[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 505538955 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.003312] x2apic enabled [ 0.004012] Switched APIC routing to physical x2apic. [ 0.005014] kvm-guest: setup PV IPIs [ 0.008000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008022] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009014] pid_max: default: 32768 minimum: 301 [ 0.010142] LSM: Security Framework initializing [ 0.011054] Yama: becoming mindful. [ 0.013038] SELinux: Initializing. [ 0.014087] *** VALIDATE selinux *** [ 0.022828] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026871] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.027163] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028113] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029118] *** VALIDATE tmpfs *** [ 0.030478] *** VALIDATE proc *** [ 0.032240] *** VALIDATE cgroup *** [ 0.033011] *** VALIDATE cgroup2 *** [ 0.035276] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.036162] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.037011] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.038033] Spectre V2 : User space: Vulnerable [ 0.039011] Speculative Store Bypass: Vulnerable [ 0.041657] debug: unmapping init [mem 0xffffffffaa859000-0xffffffffaa860fff] [ 0.043193] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044737] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045025] ... version: 2 [ 0.046013] ... bit width: 48 [ 0.047016] ... generic registers: 4 [ 0.048014] ... value mask: 0000ffffffffffff [ 0.049022] ... max period: 00007fffffffffff [ 0.050015] ... fixed-purpose events: 3 [ 0.051017] ... event mask: 000000070000000f [ 0.052286] rcu: Hierarchical SRCU implementation. [ 0.054431] smp: Bringing up secondary CPUs ... [ 0.055630] x86: Booting SMP configuration: [ 0.056040] .... node #0, CPUs: #1 #2 #3 [ 0.063691] smp: Brought up 1 node, 4 CPUs [ 0.065015] smpboot: Max logical packages: 1 [ 0.066014] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.123469] node 0 deferred pages initialised in 55ms [ 0.127748] devtmpfs: initialized [ 0.128233] x86/mm: Memory block size: 128MB [ 0.130000] gcov: version magic: 0x41383552 [ 0.134400] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.137104] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.140362] pinctrl core: initialized pinctrl subsystem [ 0.142164] [ 0.142645] ************************************************************* [ 0.144017] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.146014] ** ** [ 0.148013] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.150012] ** ** [ 0.152013] ** This means that this kernel is built to expose internal ** [ 0.154009] ** IOMMU data structures, which may compromise security on ** [ 0.156013] ** your system. ** [ 0.158013] ** ** [ 0.160009] ** If you see this message and you are not debugging the ** [ 0.162011] ** kernel, report this immediately to your vendor! ** [ 0.165026] ** ** [ 0.166104] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.168011] ************************************************************* [ 0.171650] NET: Registered protocol family 16 [ 0.172418] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.175063] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.177068] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.180078] cpuidle: using governor menu [ 0.181615] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.182310] PCI: Using configuration type 1 for base access [ 0.183101] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.189124] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.191051] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.193229] cryptd: max_cpu_qlen set to 1000 [ 0.196212] ACPI: Added _OSI(Module Device) [ 0.198014] ACPI: Added _OSI(Processor Device) [ 0.199009] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.200008] ACPI: Added _OSI(Processor Aggregator Device) [ 0.203207] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.206466] ACPI: Interpreter enabled [ 0.209061] ACPI: PM: (supports S0 S3 S4 S5) [ 0.211012] ACPI: Using IOAPIC for interrupt routing [ 0.212107] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.214315] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.221798] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.223029] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.225015] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.227069] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.229979] acpiphp: Slot [2] registered [ 0.231069] acpiphp: Slot [5] registered [ 0.231949] acpiphp: Slot [6] registered [ 0.233088] acpiphp: Slot [7] registered [ 0.234015] acpiphp: Slot [8] registered [ 0.234933] acpiphp: Slot [9] registered [ 0.236062] acpiphp: Slot [10] registered [ 0.237078] acpiphp: Slot [3] registered [ 0.238074] acpiphp: Slot [4] registered [ 0.239109] acpiphp: Slot [11] registered [ 0.240064] acpiphp: Slot [12] registered [ 0.242108] acpiphp: Slot [13] registered [ 0.243105] acpiphp: Slot [14] registered [ 0.245085] acpiphp: Slot [15] registered [ 0.246068] acpiphp: Slot [16] registered [ 0.247067] acpiphp: Slot [17] registered [ 0.248125] acpiphp: Slot [18] registered [ 0.249072] acpiphp: Slot [19] registered [ 0.250077] acpiphp: Slot [20] registered [ 0.251094] acpiphp: Slot [21] registered [ 0.252100] acpiphp: Slot [22] registered [ 0.253000] acpiphp: Slot [23] registered [ 0.253000] acpiphp: Slot [24] registered [ 0.255084] acpiphp: Slot [25] registered [ 0.256055] acpiphp: Slot [26] registered [ 0.257131] acpiphp: Slot [27] registered [ 0.258060] acpiphp: Slot [28] registered [ 0.258944] acpiphp: Slot [29] registered [ 0.260068] acpiphp: Slot [30] registered [ 0.261052] acpiphp: Slot [31] registered [ 0.262039] PCI host bridge to bus 0000:00 [ 0.262950] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.264016] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.266014] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.267029] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.270029] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.273124] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.275261] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.278992] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.280972] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.289731] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.295754] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.297013] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.298014] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.301018] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.303592] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.306872] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.309053] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.312891] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.318017] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.328959] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.334017] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.339265] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.345021] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.351017] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.366019] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.376280] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.385017] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.393016] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.412017] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.423982] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.432015] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.438016] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.457020] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.468044] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.478017] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.485016] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.514020] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.526740] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.537016] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.542014] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.556017] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.565047] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.573018] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.580023] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.594018] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.612287] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.615310] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.616222] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.618374] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.620150] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.624366] iommu: Default domain type: Passthrough [ 0.625428] SCSI subsystem initialized [ 0.626173] ACPI: bus type USB registered [ 0.628137] usbcore: registered new interface driver usbfs [ 0.629066] usbcore: registered new interface driver hub [ 0.631065] usbcore: registered new device driver usb [ 0.632182] pps_core: LinuxPPS API ver. 1 registered [ 0.633008] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.636061] PTP clock support registered [ 0.637122] EDAC MC: Ver: 3.0.0 [ 0.639112] PCI: Using ACPI for IRQ routing [ 0.640550] NetLabel: Initializing [ 0.641020] NetLabel: domain hash size = 128 [ 0.642012] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.644061] NetLabel: unlabeled traffic allowed by default [ 0.645177] vgaarb: loaded [ 0.646267] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.648009] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.654000] clocksource: Switched to clocksource kvm-clock [ 0.763194] VFS: Disk quotas dquot_6.6.0 [ 0.764858] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.766371] *** VALIDATE ramfs *** [ 0.767060] *** VALIDATE hugetlbfs *** [ 0.768162] pnp: PnP ACPI init [ 0.769714] pnp: PnP ACPI: found 6 devices [ 0.783516] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.786962] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.789257] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.791419] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.793848] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.796446] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.799233] NET: Registered protocol family 2 [ 0.801721] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.805888] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.808481] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.813345] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.817072] TCP: Hash tables configured (established 65536 bind 65536) [ 0.819753] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.822297] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.824707] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.827075] NET: Registered protocol family 1 [ 0.828872] RPC: Registered named UNIX socket transport module. [ 0.830651] RPC: Registered udp transport module. [ 0.831924] RPC: Registered tcp transport module. [ 0.833402] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.835415] NET: Registered protocol family 44 [ 0.836370] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.837826] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.839536] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.841830] PCI: CLS 0 bytes, default 64 [ 0.843572] Unpacking initramfs... [ 2.255067] debug: unmapping init [mem 0xffffa0fb7cc54000-0xffffa0fb7ffbffff] [ 2.259872] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.262260] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.264738] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.780583] Initialise system trusted keyrings [ 2.783310] Key type blacklist registered [ 2.785398] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.795386] zbud: loaded [ 2.798494] *** VALIDATE nfs *** [ 2.799833] *** VALIDATE nfs4 *** [ 2.801495] pstore: using deflate compression [ 2.805426] Platform Keyring initialized [ 2.919920] NET: Registered protocol family 38 [ 2.922056] Key type asymmetric registered [ 2.923316] Asymmetric key parser 'x509' registered [ 2.925323] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.927778] io scheduler mq-deadline registered [ 2.929944] io scheduler kyber registered [ 2.931863] io scheduler bfq registered [ 2.934262] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.937868] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.941220] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.944735] ACPI: Power Button [PWRF] [ 2.950844] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.956590] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.970218] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 2.976294] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 2.995655] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.023249] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.061361] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.065860] Non-volatile memory driver v1.3 [ 3.067706] Linux agpgart interface v0.103 [ 3.091861] virtio_blk virtio1: [vda] 146712 512-byte logical blocks (75.1 MB/71.6 MiB) [ 3.093741] vda: detected capacity change from 0 to 75116544 [ 3.106593] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.108335] vdb: detected capacity change from 0 to 1073741824 [ 3.119554] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.121974] vdc: detected capacity change from 0 to 2621440000 [ 3.134594] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.137434] vdd: detected capacity change from 0 to 2621440000 [ 3.161219] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.165164] vde: detected capacity change from 0 to 4294967296 [ 3.181981] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.183889] vdf: detected capacity change from 0 to 4294967296 [ 3.188588] libphy: Fixed MDIO Bus: probed [ 3.195870] usbcore: registered new interface driver usbserial_generic [ 3.197875] usbserial: USB Serial support registered for generic [ 3.199687] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.203573] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.205251] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.207296] mousedev: PS/2 mouse device common for all mice [ 3.210941] rtc_cmos 00:05: RTC can wake from S4 [ 3.214300] rtc_cmos 00:05: registered as rtc0 [ 3.216348] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.216555] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.219196] intel_pstate: CPU model not supported [ 3.225767] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.226578] hid: raw HID events driver (C) Jiri Kosina [ 3.231954] usbcore: registered new interface driver usbhid [ 3.233678] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.234119] usbhid: USB HID core driver [ 3.238637] drop_monitor: Initializing network drop monitor service [ 3.240849] Initializing XFRM netlink socket [ 3.242872] NET: Registered protocol family 10 [ 3.245515] Segment Routing with IPv6 [ 3.246772] NET: Registered protocol family 17 [ 3.248730] mpls_gso: MPLS GSO support [ 3.254835] RAS: Correctable Errors collector initialized. [ 3.257124] AVX version of gcm_enc/dec engaged. [ 3.258678] AES CTR mode by8 optimization enabled [ 3.330228] sched_clock: Marking stable (3330200439, 0)->(4318427146, -988226707) [ 3.333720] registered taskstats version 1 [ 3.336182] Loading compiled-in X.509 certificates [ 3.338152] zswap: loaded using pool lzo/zbud [ 3.362201] Key type big_key registered [ 3.374405] Key type encrypted registered [ 3.375911] ima: No TPM chip found, activating TPM-bypass! [ 3.378225] ima: Allocated hash algorithm: sha1 [ 3.379899] ima: No architecture policies found [ 3.381544] evm: Initialising EVM extended attributes: [ 3.383361] evm: security.selinux [ 3.384643] evm: security.ima [ 3.385595] evm: security.capability [ 3.386886] evm: HMAC attrs: 0x1 [ 3.388972] rtc_cmos 00:05: setting system clock to 2026-09-08 04:52:48 UTC (1788843168) [ 3.395644] debug: unmapping init [mem 0xffffffffab803000-0xffffffffab9fffff] [ 3.398298] debug: unmapping init [mem 0xffffffffaa582000-0xffffffffaa858fff] [ 3.406096] Write protecting the kernel read-only data: 28672k [ 3.409424] debug: unmapping init [mem 0xffffffffa8c03000-0xffffffffa8dfffff] [ 3.412112] debug: unmapping init [mem 0xffffffffa9514000-0xffffffffa95fffff] [ 3.446416] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.454462] systemd[1]: Detected virtualization kvm. [ 3.456410] systemd[1]: Detected architecture x86-64. [ 3.458249] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.486989] systemd[1]: No hostname configured. [ 3.488486] systemd[1]: Set hostname to . [ 3.490276] random: systemd: uninitialized urandom read (16 bytes read) [ 3.492455] systemd[1]: Initializing machine ID from random generator. [ 3.624364] random: systemd: uninitialized urandom read (16 bytes read) [ 3.627280] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ 3.632811] random: systemd: uninitialized urandom read (16 bytes read) [ 3.635204] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 3.639864] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ OK ] Started Memstrack Anylazing Service. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Reached target Timers. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... Starting Create Volatile Files and Directories... [ OK ] Reached target Initrd Root Device. Starting Setup Virtual Console... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Swap. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. Starting Apply Kernel Variables... [ OK ] Reached target Slices. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Static Device Nodes in /dev. Starting dracut cmdline hook... [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.252912] device-mapper: uevent: version 1.0.3 [ 4.254980] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 4.968399] random: fast init done [ 5.002416] virtio_net virtio0 ens2: renamed from eth0 [ 5.050323] scsi host0: ata_piix [ 5.089929] scsi host1: ata_piix [ 5.091564] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.093405] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.639413] dracut-initqueue[589]: RTNETLINK answers: File exists [ 9.890917] random: crng init done [ 9.892331] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.508282] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Timers. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped target Swap. [ OK ] Stopped Apply Kernel Variables. Stopping udev Kernel Device Manager... [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.759887] printk: systemd: 26 output lines suppressed due to ratelimiting [ 12.039280] SELinux: Disabled at runtime. [ 12.102706] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.113328] systemd[1]: Detected virtualization kvm. [ 12.115393] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.709983] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.713784] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.724075] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.728174] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.732387] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.741234] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.751906] systemd[1]: Mounting Huge Pages File System... Mounting Huge Pages File System... [ OK ] Created slice User and Session Slice. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Created slice system-serial\x2dgetty.slice. Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on RPCbind Server Activation Socket. [ 12.820398] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on Process Core Dump Socket. [ OK ] Reached target rpc_pipefs.target. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Listening on udev Control Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Listening on initctl Compatibility Named Pipe. Mounting POSIX Message Queue File System... Starting Apply Kernel Variables... [ OK ] Started Forward Password Requests to Wall Directory Watch. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Created slice system-getty.slice. [ OK ] Reached target Slices. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Stopped target Initrd File Systems. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. Mounting Kernel Debug File System... [ OK ] Reached target Local Encrypted Volumes. Starting Remount Root and Kernel File Systems... [ OK ] Started Journal Service. [ OK ] Mounted Huge Pages File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Kernel Debug File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 13.288816] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.702577] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.803191] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.929613] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.940846] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (6s / no limit)[ 19.087975] Key type dns_resolver registered [** ] A start job is running for Configur…-only root support (6s / no limit)[ 19.930739] NFS: Registering the id_resolver key type [ 19.948528] Key type id_resolver registered [ 19.961644] Key type id_legacy registered [*** ] A start job is running for Configur…-only root support (7s / no limit) [ *** ] A start job is running for Configur…-only root support (7s / no limit) [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Started irqbalance daemon. Starting Login Service... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... [ OK ] Started Login Service. Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started OpenSSH server daemon. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg211-server login: [ 82.320106] hrtimer: interrupt took 5061918 ns [ 85.429664] libcfs: loading out-of-tree module taints kernel. [ 85.562949] Key type ._llcrypt registered [ 85.568959] Key type .llcrypt registered [ 85.848210] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 106.760972] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 108.977522] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 109.013833] alg: No test for adler32 (adler32-zlib) [ 110.761540] Lustre: Lustre: Build Version: 2.17.57_111_g33f3974 [ 112.179197] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 114.095263] Key type lgssc registered [ 116.759177] Lustre: Echo OBD driver; http://www.lustre.org/ [ 135.962952] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 184.152734] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 202.063440] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 202.103660] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 203.499325] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 203.546757] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 203.656162] Lustre: lustre-MDT0000: new disk, initializing [ 203.804095] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 203.856062] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 209.372554] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 224.406278] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 224.560684] Lustre: 6514:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 224.617951] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 224.626740] Lustre: Skipped 1 previous similar message [ 224.750723] Lustre: lustre-MDT0001: new disk, initializing [ 224.878939] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 224.911054] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 224.918803] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 229.917796] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 235.537986] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 247.380743] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 247.873282] Lustre: lustre-OST0000: new disk, initializing [ 247.876306] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 247.880931] Lustre: 8454:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 247.972286] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 251.219083] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 251.239521] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 251.361313] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 256.859081] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 274.020035] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 274.288979] Lustre: lustre-OST0001: new disk, initializing [ 274.291355] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 274.303161] Lustre: 9528:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 274.421422] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 281.389519] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 281.741675] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 281.748861] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 282.012468] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 295.728516] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 303.573869] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 311.788458] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing check_logdir /tmp/testlogs/ [ 318.683445] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing yml_node [ 324.388313] Lustre: DEBUG MARKER: Client: 2.17.57.111 [ 327.685784] Lustre: DEBUG MARKER: MDS: 2.17.57.111 [ 331.195206] Lustre: DEBUG MARKER: OSS: 2.17.57.111 [ 333.084061] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Tue Sep 8 00:58:17 EDT 2026 [ 351.570262] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 353.464497] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 373.220244] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 373.226937] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 373.246684] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 374.244082] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 374.245900] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 374.260724] Lustre: Skipped 1 previous similar message [ 374.272211] Lustre: Skipped 3 previous similar messages [ 377.569936] Lustre: server umount lustre-MDT0000 complete [ 381.548716] LustreError: 6508:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788843546 with bad export cookie 1835235446752814127 [ 381.554415] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 381.556745] LustreError: 6508:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 384.480826] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 384.492941] Lustre: Skipped 1 previous similar message [ 384.497397] LustreError: 8590:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 384.497803] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 384.530749] LustreError: 8590:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 384.534262] Lustre: Skipped 1 previous similar message [ 386.528208] LustreError: 6521:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 386.528227] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 386.546397] LustreError: 6521:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 386.567361] Lustre: Skipped 1 previous similar message [ 388.182324] Lustre: server umount lustre-MDT0001 complete [ 393.019715] Lustre: server umount lustre-OST0000 complete [ 397.065685] Lustre: server umount lustre-OST0001 complete [ 405.105844] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 415.640798] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 455.739192] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 456.036232] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 456.059805] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 456.117324] Lustre: lustre-MDT0000: new disk, initializing [ 456.175459] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 456.189900] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 461.364880] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 471.684683] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 471.887428] Lustre: 15082:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 471.990708] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 471.997030] Lustre: Skipped 1 previous similar message [ 472.103452] Lustre: lustre-MDT0001: new disk, initializing [ 472.252344] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 472.349509] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 472.368247] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 477.204073] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 487.898820] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 493.775591] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 500.367968] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 500.774879] Lustre: lustre-OST0000: new disk, initializing [ 500.785736] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 500.797495] Lustre: 17065:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 500.934247] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 502.829100] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 502.853769] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 503.153617] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 508.727187] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 522.131811] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 527.252692] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 527.542288] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 533.646981] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 533.968199] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 539.113036] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 539.116921] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 539.137133] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 539.139019] Lustre: Skipped 2 previous similar messages [ 539.154277] Lustre: Skipped 1 previous similar message [ 549.360930] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 549.365536] Lustre: Skipped 3 previous similar messages [ 550.367231] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 550.495448] Lustre: server umount lustre-OST0000 complete [ 559.590516] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 569.311548] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 569.566575] Lustre: server umount lustre-MDT0000 complete [ 569.827380] LustreError: 15094:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 569.850692] LustreError: 15094:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 573.286187] LustreError: 15076:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788843738 with bad export cookie 1835235446752815660 [ 573.288636] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 573.314280] LustreError: 15076:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 573.770368] Lustre: server umount lustre-MDT0001 complete [ 579.474652] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 01:02:23 (1788843743) [ 586.134044] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 586.744693] LustreError: 19333:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 586.919420] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 592.118817] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 592.354366] LustreError: 19334:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 601.574286] LustreError: 19333:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 601.596606] LustreError: 19333:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 601.651768] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 607.527176] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 616.252941] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 623.779179] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 631.984641] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 639.833104] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 647.753100] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 662.383372] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 671.714849] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 671.724646] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 671.741373] Lustre: Skipped 3 previous similar messages [ 671.755778] Lustre: Skipped 6 previous similar messages [ 676.194116] Lustre: server umount lustre-MDT0000 complete [ 676.839786] LustreError: 19334:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 676.864885] LustreError: 19334:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 680.753874] LustreError: 19314:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788843845 with bad export cookie 1835235446752816542 [ 680.759978] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 680.765759] LustreError: 19314:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 681.172282] Lustre: server umount lustre-MDT0001 complete [ 695.737482] Lustre: server umount lustre-OST0000 complete [ 722.373812] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 734.034559] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 734.428653] LustreError: 23739:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 734.571173] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 734.576100] Lustre: Skipped 2 previous similar messages [ 739.416159] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 749.500052] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 754.310828] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 757.431477] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 764.517380] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 772.006752] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 783.527626] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 783.832587] Lustre: lustre-OST0001: new disk, initializing [ 783.837657] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 783.857225] Lustre: 26056:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 785.390243] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 785.402630] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 785.602205] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 791.960871] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 803.337566] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 807.673852] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 821.237710] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 821.244198] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 821.259574] Lustre: Skipped 2 previous similar messages [ 821.269469] Lustre: Skipped 3 previous similar messages [ 827.395742] Lustre: server umount lustre-MDT0000 complete [ 831.332575] LustreError: 24879:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788843996 with bad export cookie 1835235446752817389 [ 831.339459] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 831.345884] LustreError: 24879:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 831.458609] LustreError: 27280:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 831.500016] LustreError: 27280:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 831.719267] Lustre: server umount lustre-MDT0001 complete [ 838.362868] Lustre: server umount lustre-OST0000 complete [ 843.271228] Lustre: server umount lustre-OST0001 complete [ 866.914743] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 877.631764] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 878.226175] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 878.231703] Lustre: Skipped 3 previous similar messages [ 883.189568] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 892.326696] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 897.196976] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 899.817792] Lustre: 30224:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 906.439046] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 913.142752] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 922.218748] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 928.697601] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 936.147117] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 940.139087] Lustre: 32104:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 949.218744] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 949.228605] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 949.232118] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 949.232128] LustreError: Skipped 1 previous similar message [ 949.239884] Lustre: Skipped 6 previous similar messages [ 949.260778] Lustre: Skipped 9 previous similar messages [ 953.497338] Lustre: server umount lustre-MDT0000 complete [ 957.338971] LustreError: 29054:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788844122 with bad export cookie 1835235446752818684 [ 957.341144] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 957.348028] LustreError: 29054:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 957.706783] Lustre: server umount lustre-MDT0001 complete [ 971.724062] Lustre: server umount lustre-OST0000 complete [ 986.204616] Lustre: server umount lustre-OST0001 complete [ 995.546356] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 01:09:20 (1788844160) [ 1021.233156] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 1030.725303] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1030.995758] LustreError: 34910:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1031.008975] LustreError: 34910:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 8 previous similar messages [ 1035.336920] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1043.556964] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1048.541157] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1051.088976] Lustre: 36057:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1056.839660] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1063.166384] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1071.765238] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1077.717576] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1085.066189] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1088.425334] Lustre: 37934:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1095.140413] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1095.149995] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1095.169214] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1099.200474] Lustre: server umount lustre-MDT0000 complete [ 1102.614057] LustreError: 34892:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788844267 with bad export cookie 1835235446752819790 [ 1102.623526] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1102.628925] LustreError: 34892:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1103.089724] Lustre: server umount lustre-MDT0001 complete [ 1116.228518] Lustre: server umount lustre-OST0000 complete [ 1130.495456] Lustre: server umount lustre-OST0001 complete [ 1139.821493] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 01:11:44 (1788844304) [ 1156.853735] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 1168.197327] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1168.698328] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1168.705941] Lustre: Skipped 7 previous similar messages [ 1172.490731] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1180.708534] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1185.706124] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1188.687804] Lustre: 41226:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1194.687637] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1201.399900] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1210.262794] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1218.510557] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1226.032038] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1229.256690] Lustre: 43105:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1230.036936] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1230.043334] Lustre: Skipped 1 previous similar message [ 1234.407219] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1234.419909] Lustre: Skipped 4 previous similar messages [ 1239.033273] Lustre: server umount lustre-MDT0000 complete [ 1242.210317] LustreError: 42413:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788844407 with bad export cookie 1835235446752820889 [ 1242.211329] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1242.230045] LustreError: 42413:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1248.642587] Lustre: server umount lustre-MDT0001 complete [ 1255.894837] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1272.252665] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1277.868910] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1280.197539] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1286.019307] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1286.129738] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1286.137265] Lustre: Skipped 1 previous similar message [ 1292.893731] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1294.726388] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1299.874447] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1301.236221] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1306.601524] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1312.422777] Lustre: server umount lustre-MDT0000 complete [ 1315.813949] LustreError: 40057:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788844480 with bad export cookie 1835235446752822023 [ 1315.820793] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1315.832461] LustreError: 40057:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 1316.142657] Lustre: server umount lustre-MDT0001 complete [ 1330.819218] Lustre: server umount lustre-OST0000 complete [ 1332.194773] Lustre: 3649:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788844481/real 1788844481] req@ffffa0fac23f2680 x1875738128466048/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788844497 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1335.833382] Lustre: server umount lustre-OST0001 complete [ 1346.109442] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 01:15:10 (1788844510) [ 1360.546619] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 1370.817307] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1371.308183] LustreError: 47722:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1371.319677] LustreError: 47722:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 29 previous similar messages [ 1375.554482] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1383.642421] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1388.553899] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1391.724166] Lustre: 48873:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1398.607133] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1406.986349] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1416.056661] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1423.300440] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1430.423370] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1434.317679] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1445.347928] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1445.366531] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1445.378335] Lustre: Skipped 22 previous similar messages [ 1446.747525] Lustre: server umount lustre-MDT0000 complete [ 1451.024907] LustreError: 47703:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788844616 with bad export cookie 1835235446752822821 [ 1451.035738] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1451.053607] LustreError: 47703:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1451.557780] Lustre: server umount lustre-MDT0001 complete [ 1467.701387] Lustre: server umount lustre-OST0000 complete [ 1469.471490] Lustre: 3650:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788844618/real 1788844618] req@ffffa0fac4c8aa00 x1875738128520960/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788844634 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1471.808995] Lustre: server umount lustre-OST0001 complete [ 1481.200306] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 01:17:25 (1788844645) [ 1483.476605] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1485.504413] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 01:17:29 (1788844649) [ 1497.012848] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 1508.255666] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 1532.847731] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1533.130528] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1533.230175] Lustre: lustre-MDT0000: new disk, initializing [ 1533.351994] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1538.044570] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1547.248956] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1553.828700] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1554.222076] Lustre: lustre-OST0000: new disk, initializing [ 1554.227260] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1554.235134] Lustre: Skipped 1 previous similar message [ 1554.241334] Lustre: 54947:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1555.420910] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1555.436152] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1555.481425] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1560.654582] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1570.585985] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1573.501980] Lustre: DEBUG MARKER: create test files [ 1606.627361] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1606.635877] Lustre: Skipped 13 previous similar messages [ 1612.214604] Lustre: server umount lustre-OST0000 complete [ 1617.016895] Lustre: server umount lustre-MDT0000 complete [ 1630.306175] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 1634.171149] Key type lgssc unregistered [ 1634.526140] LNet: 56576:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1634.535544] LNetError: 56576:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1635.564533] LNet: Removed LNI 192.168.202.111@tcp [ 1637.148478] Key type .llcrypt unregistered [ 1637.150138] Key type ._llcrypt unregistered [ 1668.475568] Key type ._llcrypt registered [ 1668.477781] Key type .llcrypt registered [ 1668.596777] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 1669.597199] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1669.702447] alg: No test for adler32 (adler32-zlib) [ 1670.811958] Lustre: Lustre: Build Version: 2.17.57_111_g33f3974 [ 1671.093716] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 1672.775178] Key type lgssc registered [ 1674.118587] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1682.575180] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1682.609547] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1684.327204] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1689.150851] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1696.508332] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1703.663426] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1703.887094] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1709.054644] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:99 to 0x240000400:129) [ 1709.141583] Lustre: 59311:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1709.162436] Lustre: 59311:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1709.170214] Lustre: 59311:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1709.175149] Lustre: 59311:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1709.182870] Lustre: 59311:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1709.199164] Lustre: 59311:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1710.375887] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1717.599302] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1722.374851] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1726.727684] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1739.748893] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1739.755471] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1739.763610] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1743.498548] Lustre: server umount lustre-OST0000 complete [ 1748.298157] Lustre: server umount lustre-MDT0000 complete [ 1764.742899] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 1768.207509] Key type lgssc unregistered [ 1768.546380] LNet: 60920:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1768.557377] LNetError: 60920:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1768.571700] LNet: Removed LNI 192.168.202.111@tcp [ 1769.549176] Key type .llcrypt unregistered [ 1769.551453] Key type ._llcrypt unregistered [ 1803.407247] Key type ._llcrypt registered [ 1803.410341] Key type .llcrypt registered [ 1803.532710] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 1804.885676] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1805.001071] alg: No test for adler32 (adler32-zlib) [ 1806.282617] Lustre: Lustre: Build Version: 2.17.57_111_g33f3974 [ 1806.568200] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 1808.225282] Key type lgssc registered [ 1809.855823] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1818.601933] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1818.664064] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1820.137214] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1825.041525] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1833.502661] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1840.475246] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1840.798138] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1844.921967] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1845.045807] Lustre: 63754:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1845.059538] Lustre: 63754:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1845.074666] Lustre: 63754:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1845.084328] Lustre: 63754:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1845.090788] Lustre: 63754:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1845.103064] Lustre: 63754:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1847.602598] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1855.637598] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1860.958928] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1901.540428] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1901.584197] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1904.398218] Lustre: server umount lustre-OST0000 complete [ 1909.282401] Lustre: server umount lustre-MDT0000 complete [ 1924.952891] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 1927.923341] Key type lgssc unregistered [ 1928.274295] LNet: 65344:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1928.285352] LNetError: 65344:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1928.298860] LNet: Removed LNI 192.168.202.111@tcp [ 1929.845165] Key type .llcrypt unregistered [ 1929.850822] Key type ._llcrypt unregistered [ 1951.573038] Key type ._llcrypt registered [ 1951.576126] Key type .llcrypt registered [ 1951.691721] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 1967.518810] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 1968.744239] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1968.864958] alg: No test for adler32 (adler32-zlib) [ 1970.058681] Lustre: Lustre: Build Version: 2.17.57_111_g33f3974 [ 1970.255242] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 1971.935266] Key type lgssc registered [ 1973.181423] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2009.419572] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2009.449660] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2010.775257] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2010.823489] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2010.939507] Lustre: lustre-MDT0000: new disk, initializing [ 2011.174127] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2011.226026] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2016.142561] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2026.938458] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2027.074630] Lustre: 68494:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2027.145417] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2027.149400] Lustre: Skipped 1 previous similar message [ 2027.217382] Lustre: lustre-MDT0001: new disk, initializing [ 2027.328347] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2027.376984] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2027.391865] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2031.021467] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2040.465936] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2045.464675] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2051.729480] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2052.089547] Lustre: lustre-OST0000: new disk, initializing [ 2052.095692] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2052.100947] Lustre: 70462:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2052.205982] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2058.521929] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2062.414418] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2062.445653] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2062.621967] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2068.553128] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2074.563245] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2074.816383] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2080.481471] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2080.826402] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2088.419823] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2088.435754] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2088.436684] LustreError: Skipped 1 previous similar message [ 2088.447558] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2093.544133] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2093.548307] Lustre: Skipped 2 previous similar messages [ 2098.143549] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2098.471992] Lustre: server umount lustre-OST0000 complete [ 2103.777493] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2103.791711] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2103.803343] Lustre: Skipped 2 previous similar messages [ 2103.815534] Lustre: Skipped 1 previous similar message [ 2108.901812] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2108.906851] Lustre: Skipped 1 previous similar message [ 2109.852929] Lustre: server umount lustre-MDT0000 complete [ 2113.825203] LustreError: 68488:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788845278 with bad export cookie 11669056456071731466 [ 2113.834498] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2113.836322] LustreError: 68488:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2114.317395] Lustre: server umount lustre-MDT0001 complete [ 2124.335782] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 01:28:08 (1788845288) [ 2139.179916] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 2182.948873] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 2193.323338] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 2236.165221] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2236.623959] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2236.657934] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2236.800541] Lustre: lustre-MDT0000: new disk, initializing [ 2236.937551] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2236.966371] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2243.314411] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2255.775579] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2255.929076] Lustre: 76508:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2255.985321] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2255.988416] Lustre: Skipped 1 previous similar message [ 2256.096549] Lustre: lustre-MDT0001: new disk, initializing [ 2256.288080] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2256.366861] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2256.396246] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2260.365439] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2269.793882] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2275.781174] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2283.142113] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2283.550255] Lustre: lustre-OST0000: new disk, initializing [ 2283.554908] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2283.568798] Lustre: 78474:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2283.651650] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2290.703064] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2291.822797] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2291.842749] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2291.999988] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2302.698797] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2308.218912] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2308.407540] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2312.840213] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2313.052804] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2317.801046] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2317.807695] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2317.813607] Lustre: Skipped 1 previous similar message [ 2317.814813] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2317.846639] LustreError: Skipped 1 previous similar message [ 2321.195342] Lustre: server umount lustre-OST0000 complete [ 2328.032826] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2328.038634] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2328.049064] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2328.049071] Lustre: Skipped 1 previous similar message [ 2328.083090] Lustre: Skipped 1 previous similar message [ 2339.299953] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2339.460642] Lustre: server umount lustre-MDT0000 complete [ 2343.289047] LustreError: 76499:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788845508 with bad export cookie 11669056456071732348 [ 2343.297036] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2343.301093] LustreError: 76499:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2343.752455] Lustre: server umount lustre-MDT0001 complete [ 2351.702880] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 01:31:56 (1788845516) [ 2358.379038] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2359.068293] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2363.828314] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2364.389351] LustreError: 80891:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2364.418083] LustreError: 80891:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2369.509094] LustreError: 80890:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2371.612738] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2376.837980] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2385.602901] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2391.664708] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2401.758330] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2402.371650] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2402.385571] Lustre: Skipped 1 previous similar message [ 2410.371038] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2418.847944] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2424.109916] Lustre: 83561:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting 352bde5a-7e25-400d-85fa-c49ccaa1088c at adminstrative request [ 2428.414658] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2428.447782] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2438.635377] Lustre: MGS: Client 352bde5a-7e25-400d-85fa-c49ccaa1088c (at 0@lo) reconnecting [ 2438.649553] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xa1f0d10580225fee to 0xa1f0d1058022628e [ 2438.674195] Lustre: MGC192.168.202.111@tcp: Connection restored to 0@lo (at 0@lo) [ 2461.473742] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2461.701399] Lustre: lustre-OST0001: new disk, initializing [ 2461.705339] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2461.709958] Lustre: 83954:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2467.625204] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2467.967106] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2467.981285] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2468.143052] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2476.982371] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2483.706701] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2483.712830] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2483.716466] LustreError: Skipped 1 previous similar message [ 2483.722397] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2483.729154] Lustre: Skipped 1 previous similar message [ 2483.746277] Lustre: Skipped 6 previous similar messages [ 2486.120992] Lustre: server umount lustre-OST0001 complete [ 2488.820974] LustreError: 82712:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2488.863072] LustreError: 82712:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2489.825895] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2491.878576] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2491.890859] Lustre: Skipped 1 previous similar message [ 2493.923172] LustreError: 83582:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2493.968828] LustreError: 83582:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2496.063195] Lustre: server umount lustre-OST0000 complete [ 2506.802456] Lustre: server umount lustre-MDT0000 complete [ 2509.285855] LustreError: 80891:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2509.312792] LustreError: 80891:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2510.548600] LustreError: 80872:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788845675 with bad export cookie 11669056456071733902 [ 2510.553170] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2510.568644] LustreError: 80872:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2510.979195] Lustre: server umount lustre-MDT0001 complete [ 2518.492471] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 01:34:43 (1788845683) [ 2519.944106] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2521.619856] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 01:34:46 (1788845686) [ 2523.589758] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2525.786293] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 01:34:49 (1788845689) [ 2527.200552] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2529.007578] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 01:34:53 (1788845693) [ 2530.438035] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2531.982446] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 01:34:56 (1788845696) [ 2533.342557] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2538.504138] loop: module loaded [ 2538.529921] loop0: detected capacity change from 0 to 4294967296 [ 2538.587283] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2539.974547] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2543.617845] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2547.352692] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2574.529612] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2574.575992] LustreError: 88082:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2574.586339] LustreError: 88082:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2574.597090] LustreError: 88082:0:(tgt_mount.c:2582:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2574.608849] LustreError: 88082:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2594.363648] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 2605.735535] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 2642.305695] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2642.539258] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2642.578092] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2642.637246] Lustre: lustre-MDT0000: new disk, initializing [ 2642.704085] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2642.707157] Lustre: Skipped 1 previous similar message [ 2642.720564] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2647.175800] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2657.156483] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2657.256789] Lustre: 90950:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2657.513522] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2661.157926] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2670.408927] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2675.365443] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2681.669312] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2681.890250] Lustre: 92917:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2687.423214] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2692.197914] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2692.355846] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2698.059334] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2702.791605] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2703.053465] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2707.944836] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2708.166483] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2713.061911] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2713.077800] LustreError: Skipped 1 previous similar message [ 2713.086267] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2713.102702] Lustre: Skipped 2 previous similar messages [ 2713.111016] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2713.117892] Lustre: Skipped 7 previous similar messages [ 2724.319367] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2724.554727] Lustre: server umount lustre-OST0000 complete [ 2734.562109] Lustre: server umount lustre-MDT0000 complete [ 2738.301489] LustreError: 90941:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788845903 with bad export cookie 11669056456071734511 [ 2738.302050] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2738.327189] LustreError: 90941:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2738.768088] Lustre: server umount lustre-MDT0001 complete [ 2746.621045] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 01:38:31 (1788845911) [ 2753.032538] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2753.430506] LustreError: 95331:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2757.702738] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2765.541586] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2769.539326] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2776.343465] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2781.402669] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2787.356255] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2787.643874] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2787.652909] Lustre: Skipped 4 previous similar messages [ 2793.587520] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2800.577949] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2804.692351] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2804.845463] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2809.352593] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2809.496859] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2816.819478] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2817.039657] Lustre: lustre-OST0001: new disk, initializing [ 2817.042151] Lustre: Skipped 2 previous similar messages [ 2817.045679] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2817.049567] Lustre: Skipped 3 previous similar messages [ 2817.053915] Lustre: 98700:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2823.382686] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2823.402591] Lustre: Skipped 2 previous similar messages [ 2823.424574] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2823.598380] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2825.618340] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2836.920147] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2842.741664] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2843.039966] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2848.073438] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2848.300642] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2869.283742] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2891.348206] Lustre: Failing over lustre-MDT0000 [ 2891.824198] Lustre: server umount lustre-MDT0000 complete [ 2891.960261] LustreError: 95327:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.202.11@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2891.973664] LustreError: 95327:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2893.792884] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2893.800302] LustreError: Skipped 1 previous similar message [ 2893.806132] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2893.816986] Lustre: Skipped 3 previous similar messages [ 2911.713808] Lustre: 66718:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788846060/real 1788846060] req@000000003aa04d0b x1875740076556288/t0(0) o400->MGC192.168.202.111@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1788846076 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2911.745149] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2918.448675] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2918.451619] LDISKFS-fs (dm-0): recovery complete [ 2918.457993] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2920.954022] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0xa1f0d10580232cfb [ 2920.962070] Lustre: MGC192.168.202.111@tcp: Connection restored to 0@lo (at 0@lo) [ 2921.194183] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2922.656058] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2922.657732] Lustre: 95328:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2925.797495] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2926.576306] Lustre: 96059:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2926.582587] Lustre: 96059:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 2926.598824] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2926.677696] LustreError: 101558:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2927.000352] LustreError: 101558:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2927.333319] LustreError: 101558:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2927.340265] LustreError: 101558:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2927.668429] LustreError: 101558:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2927.677263] LustreError: 101558:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2927.682179] Lustre: 101558:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2927.694212] Lustre: 101558:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 2928.345727] LustreError: 101558:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2928.358988] LustreError: 101558:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2928.679135] LustreError: 101558:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2928.687263] LustreError: 101558:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2929.969083] Lustre: 101558:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2929.987026] Lustre: 101558:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 6 previous similar messages [ 2930.645842] LustreError: 101558:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2930.669644] LustreError: 101558:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2930.993051] LustreError: 101558:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2930.997362] LustreError: 101558:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2934.186344] Lustre: 101558:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2934.205285] Lustre: 101558:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 2934.875515] LustreError: 101558:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2934.896333] LustreError: 101558:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2935.223139] LustreError: 101558:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2935.234648] LustreError: 101558:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2942.295655] Lustre: 101558:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2942.311824] Lustre: 101558:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 24 previous similar messages [ 2942.975616] LustreError: 101558:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2942.997824] LustreError: 101558:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2943.327173] LustreError: 101558:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2943.330690] LustreError: 101558:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2958.592062] Lustre: 101558:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2958.610027] Lustre: 101558:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 50 previous similar messages [ 2959.281305] LustreError: 101558:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2959.294046] LustreError: 101558:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2959.640487] LustreError: 101558:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2959.652895] LustreError: 101558:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2982.087210] Lustre: 101558:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2982.094845] Lustre: 101558:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2982.104619] Lustre: 101558:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 72d82406-4992-4670-b0f6-e22fcb3635a1@192.168.202.11@tcp [ 2982.113900] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2982.116709] Lustre: 101558:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@00000000ecb222ec x1875740060618240/t0(8589934769) o101->72d82406-4992-4670-b0f6-e22fcb3635a1@192.168.202.11@tcp:497/0 lens 592/0 e 0 to 0 dl 1788846157 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2982.141766] Lustre: lustre-MDT0000: Denying connection for new client 72d82406-4992-4670-b0f6-e22fcb3635a1 (at 192.168.202.11@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2982.165794] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2982.183393] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2982.218843] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2982.223946] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2982.229091] Lustre: Skipped 2 previous similar messages [ 2982.266782] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:45 to 0x2c0000401:545) [ 2982.270779] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:77 to 0x280000401:577) [ 2998.254776] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2998.259271] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2998.275580] Lustre: Skipped 3 previous similar messages [ 2998.281568] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2998.288495] Lustre: Skipped 8 previous similar messages [ 3002.210858] Lustre: server umount lustre-OST0000 complete [ 3003.387412] LustreError: 97153:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3003.411941] LustreError: 97153:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 40 previous similar messages [ 3020.255324] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3020.485104] Lustre: server umount lustre-OST0001 complete [ 3030.815302] Lustre: server umount lustre-MDT0000 complete [ 3034.517560] LustreError: 98698:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788846199 with bad export cookie 11669056456071785723 [ 3034.528155] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3034.532740] LustreError: 98698:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3034.964198] Lustre: server umount lustre-MDT0001 complete [ 3043.701662] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 01:43:28 (1788846208) [ 3051.373193] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3051.383426] Lustre: *** cfs_fail_loc=197, val=0*** [ 3066.975602] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3072.098117] LustreError: 103532:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.111@tcp: failed processing log, type 4: rc = -110 [ 3077.267182] Lustre: *** cfs_fail_loc=197, val=0*** [ 3077.275822] Lustre: Skipped 65 previous similar messages [ 3097.888530] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3097.898046] Lustre: Skipped 2 previous similar messages [ 3105.190513] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3113.679429] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3115.972869] Lustre: server umount lustre-OST0000 complete [ 3126.158445] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 01:44:50 (1788846290) [ 3150.717748] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3151.406860] LustreError: 105152:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3151.422965] LustreError: 105152:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 8 previous similar messages [ 3157.018886] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3168.429147] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3172.696213] LustreError: 105183:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3172.705718] LustreError: 105183:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3172.718795] LustreError: 105183:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 21, retries 0, failed: rc = -5 [ 3173.176594] Lustre: server umount lustre-MDT0000 complete [ 3211.398795] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 3221.839623] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 3259.828501] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3260.219948] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3260.313815] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3260.425759] Lustre: lustre-MDT0000: new disk, initializing [ 3260.563205] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3266.763767] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3280.009024] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3280.163935] Lustre: 109359:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3280.560125] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3284.781332] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3293.829549] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3300.352285] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3308.715238] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3308.954803] Lustre: 111328:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3315.761545] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3318.003656] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3325.799636] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3331.700975] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3331.978539] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3338.227599] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3338.446958] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3343.857506] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3343.862871] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3343.878369] LustreError: Skipped 3 previous similar messages [ 3343.892156] Lustre: Skipped 5 previous similar messages [ 3343.902808] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3343.912993] Lustre: Skipped 10 previous similar messages [ 3355.103832] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3355.464097] Lustre: server umount lustre-OST0000 complete [ 3365.335894] Lustre: server umount lustre-MDT0000 complete [ 3369.297279] LustreError: 109350:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788846534 with bad export cookie 11669056456071796279 [ 3369.301807] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3369.308973] LustreError: 109350:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3369.761598] Lustre: server umount lustre-MDT0001 complete [ 3378.708535] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 01:49:03 (1788846543) [ 3387.768613] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 3391.251297] Key type lgssc unregistered [ 3391.519391] LNet: 113818:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3391.523738] LNetError: 113818:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3391.547892] LNet: Removed LNI 192.168.202.111@tcp [ 3392.603621] Key type .llcrypt unregistered [ 3392.606065] Key type ._llcrypt unregistered [ 3411.373283] Key type ._llcrypt registered [ 3411.375401] Key type .llcrypt registered [ 3411.517174] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 3427.895879] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 3429.404298] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3429.452067] alg: No test for adler32 (adler32-zlib) [ 3430.729764] Lustre: Lustre: Build Version: 2.17.57_111_g33f3974 [ 3431.079694] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 3432.783284] Key type lgssc registered [ 3434.099085] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3473.995681] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3474.034740] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3475.376560] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3475.406384] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3475.464724] Lustre: lustre-MDT0000: new disk, initializing [ 3475.543259] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3475.555549] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3480.281967] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3490.017383] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3490.361289] Lustre: lustre-OST0000: new disk, initializing [ 3490.365021] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3490.368962] Lustre: Skipped 1 previous similar message [ 3490.378040] Lustre: 117986:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3490.501426] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3496.000580] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3496.013966] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3496.066088] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3498.569806] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3511.439051] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3511.651255] Lustre: lustre-OST0001: new disk, initializing [ 3511.655221] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3511.661575] Lustre: 118848:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3511.733108] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3517.944421] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3521.151870] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3521.162304] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3521.205618] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3531.235151] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3531.242498] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3531.256799] Lustre: Skipped 1 previous similar message [ 3531.273144] Lustre: Skipped 1 previous similar message [ 3533.701477] Lustre: server umount lustre-MDT0000 complete [ 3539.885247] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3544.209503] LustreError: 117174:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788846709 with bad export cookie 9942180726277531301 [ 3544.222729] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3544.229413] LustreError: 117174:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3544.329810] Lustre: server umount lustre-OST0000 complete [ 3548.205802] Lustre: server umount lustre-OST0001 complete [ 3556.273031] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 01:52:00 (1788846720) [ 3579.332869] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 01:52:23 (1788846743) [ 3589.451805] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 3610.760759] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 01:52:54 (1788846774) [ 3651.316548] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 3662.517166] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 3703.527093] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3703.788677] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3703.822992] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3703.950896] Lustre: lustre-MDT0000: new disk, initializing [ 3704.118654] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3704.144185] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3708.980735] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3722.005878] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3722.163213] Lustre: 126616:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3722.329765] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3722.345238] Lustre: Skipped 1 previous similar message [ 3722.543292] Lustre: lustre-MDT0001: new disk, initializing [ 3722.716295] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3722.773609] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3722.794390] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3729.567334] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3740.292498] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3746.591687] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3753.792362] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3754.052558] Lustre: lustre-OST0000: new disk, initializing [ 3754.056274] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3754.068172] Lustre: 128592:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3754.173462] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3759.691728] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3759.705163] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3759.820551] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3760.275957] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3769.693481] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3774.653341] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3774.931846] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3779.364992] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3779.581746] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3785.698198] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3785.700359] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3785.716212] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3785.723473] Lustre: Skipped 1 previous similar message [ 3785.753921] LustreError: Skipped 1 previous similar message [ 3788.254517] Lustre: server umount lustre-OST0000 complete [ 3794.402330] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3794.405605] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3794.430344] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3794.437412] Lustre: Skipped 1 previous similar message [ 3799.041795] Lustre: server umount lustre-MDT0000 complete [ 3803.559684] LustreError: 126610:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788846968 with bad export cookie 9942180726277532526 [ 3803.570675] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3803.571343] LustreError: 126610:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3804.116374] Lustre: server umount lustre-MDT0001 complete [ 3813.734972] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 01:56:17 (1788846977) [ 3821.884159] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3822.488983] LustreError: 131005:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3822.521343] LustreError: 131005:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 3822.564242] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3827.682434] LustreError: 131006:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3828.228446] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3832.806472] LustreError: 131005:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3837.834050] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3837.922767] LustreError: 131006:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3844.630550] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3853.040151] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3858.866068] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3866.090833] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3866.484524] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3866.490892] Lustre: Skipped 1 previous similar message [ 3873.888621] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3882.176079] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3890.580578] Lustre: *** cfs_fail_loc=159, val=0*** [ 3906.293751] Lustre: lustre-MDT0000: Client 65e1b87c-b3df-4ea9-b6f3-611b4b7d0d64 (at 192.168.202.11@tcp) reconnecting [ 3908.199223] Lustre: *** cfs_fail_loc=159, val=0*** [ 3908.212616] Lustre: Skipped 5 previous similar messages [ 3924.691661] Lustre: lustre-MDT0000: Client 65e1b87c-b3df-4ea9-b6f3-611b4b7d0d64 (at 192.168.202.11@tcp) reconnecting [ 3930.081327] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3930.094096] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3930.107512] Lustre: Skipped 1 previous similar message [ 3930.111134] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3930.111146] Lustre: Skipped 2 previous similar messages [ 3931.646733] Lustre: server umount lustre-OST0000 complete [ 3940.330575] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3940.339705] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3940.349347] Lustre: Skipped 2 previous similar messages [ 3940.381212] Lustre: Skipped 2 previous similar messages [ 3943.625991] Lustre: server umount lustre-MDT0000 complete [ 3945.443818] LustreError: 131000:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3945.475514] LustreError: 131000:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3948.534763] LustreError: 130986:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788847113 with bad export cookie 9942180726277533408 [ 3948.541346] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3948.555813] LustreError: 130986:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3948.960448] Lustre: server umount lustre-MDT0001 complete [ 3962.435785] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 3965.489642] Key type lgssc unregistered [ 3965.858071] LNet: 134779:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3965.864139] LNetError: 134779:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3966.889811] LNet: Removed LNI 192.168.202.111@tcp [ 3967.782613] Key type .llcrypt unregistered [ 3967.784174] Key type ._llcrypt unregistered [ 3994.513501] Key type ._llcrypt registered [ 3994.516581] Key type .llcrypt registered [ 3994.716916] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 3995.784717] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3995.865348] alg: No test for adler32 (adler32-zlib) [ 3997.183091] Lustre: Lustre: Build Version: 2.17.57_111_g33f3974 [ 3997.433506] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 3999.199987] Key type lgssc registered [ 4000.682454] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4009.207296] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4009.226232] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4010.724597] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4014.949394] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4020.211166] LustreError: 136195:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4020.227318] LustreError: 136195:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4024.259301] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4024.597750] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4029.039568] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4036.963423] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4043.748127] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4050.478075] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4050.821937] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4055.921407] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4057.281873] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4064.961559] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4077.397716] Lustre: *** cfs_fail_loc=159, val=0*** [ 4093.653719] Lustre: lustre-MDT0000: Client d60a9aea-6492-4d1b-a55d-7b10f6f77eb6 (at 192.168.202.11@tcp) reconnecting [ 4096.545490] Lustre: *** cfs_fail_loc=159, val=0*** [ 4113.118413] Lustre: lustre-MDT0001: Client d60a9aea-6492-4d1b-a55d-7b10f6f77eb6 (at 192.168.202.11@tcp) reconnecting [ 4114.922367] Lustre: *** cfs_fail_loc=159, val=0*** [ 4114.923897] Lustre: Skipped 3 previous similar messages [ 4130.517735] Lustre: lustre-MDT0001: Client d60a9aea-6492-4d1b-a55d-7b10f6f77eb6 (at 192.168.202.11@tcp) reconnecting [ 4147.283627] Lustre: 139266:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4150.576944] Lustre: *** cfs_fail_loc=159, val=0*** [ 4150.578636] Lustre: Skipped 4 previous similar messages [ 4166.401876] Lustre: lustre-MDT0000: Client 2347048b-d864-441a-b0a1-e2014d68e069 (at 192.168.202.11@tcp) reconnecting [ 4168.304054] Lustre: *** cfs_fail_loc=159, val=0*** [ 4168.310905] Lustre: Skipped 13 previous similar messages [ 4184.819368] Lustre: lustre-MDT0000: Client 2347048b-d864-441a-b0a1-e2014d68e069 (at 192.168.202.11@tcp) reconnecting [ 4186.521774] Lustre: 139557:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4207.594402] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4207.596531] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4207.596662] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4207.614104] Lustre: Skipped 1 previous similar message [ 4211.491845] Lustre: server umount lustre-OST0000 complete [ 4217.824597] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4217.846966] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4217.852365] Lustre: Skipped 2 previous similar messages [ 4221.757987] Lustre: server umount lustre-MDT0000 complete [ 4222.951722] LustreError: 136194:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4222.965317] LustreError: 136194:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4225.761639] LustreError: 139142:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788847390 with bad export cookie 8436558610922466644 [ 4225.764609] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4225.768972] LustreError: 139142:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4226.187415] Lustre: server umount lustre-MDT0001 complete [ 4240.040780] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 4242.846094] Key type lgssc unregistered [ 4243.184332] LNet: 140635:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4243.191276] LNetError: 140635:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4243.210806] LNet: Removed LNI 192.168.202.111@tcp [ 4244.665077] Key type .llcrypt unregistered [ 4244.674747] Key type ._llcrypt unregistered [ 4272.128389] Key type ._llcrypt registered [ 4272.129862] Key type .llcrypt registered [ 4272.254133] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 4273.538084] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4273.836212] alg: No test for adler32 (adler32-zlib) [ 4275.087584] Lustre: Lustre: Build Version: 2.17.57_111_g33f3974 [ 4275.451455] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 4277.199168] Key type lgssc registered [ 4278.816731] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4289.725418] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4289.741786] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4291.318096] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4296.388852] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4301.804680] LustreError: 142045:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4301.814788] LustreError: 142045:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4306.121081] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4306.475211] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4311.217851] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4319.360544] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4325.562566] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4332.223038] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4332.743273] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4336.844599] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4339.737764] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4347.783492] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4351.151382] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4357.713248] Lustre: 144627:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4360.410374] Lustre: 144675:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4362.084720] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4362.968548] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4366.612320] Lustre: 144821:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4367.585793] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4368.515629] Lustre: 144919:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4377.061479] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4377.064621] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4377.079774] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4377.094232] Lustre: Skipped 1 previous similar message [ 4382.187348] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4382.197271] Lustre: Skipped 2 previous similar messages [ 4387.295754] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4387.297883] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4387.309059] Lustre: Skipped 1 previous similar message [ 4387.503914] Lustre: server umount lustre-OST0000 complete [ 4392.416812] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4392.444103] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4392.463732] Lustre: Skipped 1 previous similar message [ 4397.538639] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4397.549849] Lustre: Skipped 1 previous similar message [ 4398.485330] Lustre: server umount lustre-MDT0000 complete [ 4402.656595] LustreError: 142040:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4402.689671] LustreError: 142040:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4402.864629] LustreError: 142025:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788847567 with bad export cookie 18329242814817915667 [ 4402.865312] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4402.878253] LustreError: 142025:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4403.586911] Lustre: server umount lustre-MDT0001 complete [ 4418.070970] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 4422.794761] Key type lgssc unregistered [ 4423.184347] LNet: 145982:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4423.192973] LNetError: 145982:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4423.249625] LNet: Removed LNI 192.168.202.111@tcp [ 4424.323686] Key type .llcrypt unregistered [ 4424.328558] Key type ._llcrypt unregistered [ 4448.862948] Key type ._llcrypt registered [ 4448.866358] Key type .llcrypt registered [ 4448.923700] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 4449.742854] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4449.803736] alg: No test for adler32 (adler32-zlib) [ 4450.902692] Lustre: Lustre: Build Version: 2.17.57_111_g33f3974 [ 4451.133504] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 4452.794222] Key type lgssc registered [ 4454.314543] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4462.830592] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4462.851717] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4464.318168] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4469.250818] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4474.852898] LustreError: 147394:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4474.884460] LustreError: 147394:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4479.737685] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4479.968892] LustreError: 147395:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4480.343120] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4486.439601] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4495.495557] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4502.676137] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4511.633361] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4512.081626] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4519.419440] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4525.426311] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4529.157974] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4534.672226] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4539.263190] Lustre: *** cfs_fail_loc=159, val=0*** [ 4555.459205] Lustre: lustre-MDT0000: Client fcc533cc-906f-4fc5-a5a4-a8d667e589c9 (at 192.168.202.11@tcp) reconnecting [ 4575.723787] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4575.733875] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4575.746897] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4579.302587] Lustre: server umount lustre-OST0000 complete [ 4585.957914] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4585.978732] Lustre: Skipped 1 previous similar message [ 4585.986331] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4585.998277] Lustre: Skipped 2 previous similar messages [ 4590.593793] Lustre: server umount lustre-MDT0000 complete [ 4591.076472] LustreError: 148497:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4591.106609] LustreError: 148497:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4594.794390] LustreError: 147376:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788847759 with bad export cookie 8887414375684512792 [ 4594.798231] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4594.800556] LustreError: 147376:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4595.238117] Lustre: server umount lustre-MDT0001 complete [ 4606.704242] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 4610.165541] Key type lgssc unregistered [ 4610.460763] LNet: 151061:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4610.470563] LNetError: 151061:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4610.501430] LNet: Removed LNI 192.168.202.111@tcp [ 4611.476960] Key type .llcrypt unregistered [ 4611.480780] Key type ._llcrypt unregistered [ 4641.420794] Key type ._llcrypt registered [ 4641.435119] Key type .llcrypt registered [ 4641.678147] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 4643.578139] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4643.664248] alg: No test for adler32 (adler32-zlib) [ 4644.835783] Lustre: Lustre: Build Version: 2.17.57_111_g33f3974 [ 4645.313226] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 4647.167250] Key type lgssc registered [ 4648.434445] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4657.626408] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4657.674310] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4659.455093] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4662.654533] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4668.898656] LustreError: 152475:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4668.916960] LustreError: 152475:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4672.636658] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4673.062404] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4677.806696] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4685.451468] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4690.860643] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4696.775634] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4697.105257] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4698.166234] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4702.666459] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4704.362385] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4710.100273] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4731.196145] Lustre: lustre-MDT0001: evicting 391284d6-c7b8-438b-86fb-1e7fb4193427 (at 192.168.202.11@tcp) by administrative request [ 4731.206237] Lustre: Skipped 1 previous similar message [ 4760.545266] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4760.553704] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4760.560670] Lustre: Skipped 1 previous similar message [ 4760.571872] Lustre: Skipped 1 previous similar message [ 4765.664619] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4770.787359] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4770.791199] Lustre: Skipped 1 previous similar message [ 4774.367329] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4774.653580] Lustre: server umount lustre-OST0000 complete [ 4780.512298] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4780.517596] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4780.535300] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4784.858850] Lustre: server umount lustre-MDT0000 complete [ 4786.146893] LustreError: 153483:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4786.166948] LustreError: 153483:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4788.268694] LustreError: 152454:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788847953 with bad export cookie 6301720799810474824 [ 4788.274653] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4788.276696] LustreError: 152454:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4788.627790] Lustre: server umount lustre-MDT0001 complete [ 4797.584847] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 4799.764941] Key type lgssc unregistered [ 4800.042151] LNet: 156540:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4800.051286] LNetError: 156540:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4800.063715] LNet: Removed LNI 192.168.202.111@tcp [ 4800.820647] Key type .llcrypt unregistered [ 4800.822721] Key type ._llcrypt unregistered [ 4875.388113] Key type ._llcrypt registered [ 4875.391389] Key type .llcrypt registered [ 4875.488665] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 4876.614311] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4876.651431] alg: No test for adler32 (adler32-zlib) [ 4877.786372] Lustre: Lustre: Build Version: 2.17.57_111_g33f3974 [ 4878.139857] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 4879.823399] Key type lgssc registered [ 4880.907656] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4888.465989] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4888.476653] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4890.001042] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4894.828061] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4900.331169] LustreError: 159024:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4900.346871] LustreError: 159024:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4903.394408] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4903.840806] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4908.195767] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4915.444233] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4922.537624] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4931.667322] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4932.351026] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4935.426372] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4941.453922] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4948.763685] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4951.759279] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4961.248742] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4961.259404] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4961.269759] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4964.322406] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4964.322706] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4964.325585] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4966.860974] Lustre: server umount lustre-OST0000 complete [ 4974.561299] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4974.566403] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4974.585331] Lustre: Skipped 1 previous similar message [ 4974.596452] Lustre: Skipped 2 previous similar messages [ 4977.706312] Lustre: server umount lustre-MDT0000 complete [ 4979.688750] LustreError: 159754:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4979.719741] LustreError: 159754:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4981.485083] LustreError: 161614:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788848146 with bad export cookie 12917501620688942401 [ 4981.489018] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4981.494020] LustreError: 161614:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4981.843633] Lustre: server umount lustre-MDT0001 complete [ 4994.603662] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 4996.829239] Key type lgssc unregistered [ 4997.221503] LNet: 162569:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4997.231852] LNetError: 162569:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4997.253301] LNet: Removed LNI 192.168.202.111@tcp [ 4998.318108] Key type .llcrypt unregistered [ 4998.320478] Key type ._llcrypt unregistered [ 5026.552237] Key type ._llcrypt registered [ 5026.565133] Key type .llcrypt registered [ 5026.771994] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 5028.651763] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5028.810062] alg: No test for adler32 (adler32-zlib) [ 5029.907379] Lustre: Lustre: Build Version: 2.17.57_111_g33f3974 [ 5030.244629] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 5031.983321] Key type lgssc registered [ 5033.453800] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5048.029191] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 02:16:52 (1788848212) [ 5062.885056] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 5073.017278] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 5109.724039] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5109.746651] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5111.087715] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5111.111899] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5111.194929] Lustre: lustre-MDT0000: new disk, initializing [ 5111.304889] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5111.329977] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5115.692277] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 5126.791916] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5126.943193] Lustre: 166828:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5127.033619] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5127.043556] Lustre: Skipped 1 previous similar message [ 5127.161601] Lustre: lustre-MDT0001: new disk, initializing [ 5127.273694] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5127.312547] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5127.322641] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5131.895348] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 5140.264454] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5146.674739] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5153.589291] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5153.919763] Lustre: lustre-OST0000: new disk, initializing [ 5153.924296] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5153.933137] Lustre: 168799:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5154.014375] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5160.845978] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 5162.007967] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5162.033966] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5162.208270] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5170.538309] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5182.944273] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5182.945708] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5182.962920] LustreError: Skipped 1 previous similar message [ 5182.964770] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5182.990021] Lustre: Skipped 1 previous similar message [ 5183.003163] Lustre: Skipped 1 previous similar message [ 5186.357768] Lustre: server umount lustre-OST0000 complete [ 5193.186050] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5193.186945] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5193.199606] Lustre: Skipped 1 previous similar message [ 5197.128403] Lustre: server umount lustre-MDT0000 complete [ 5200.993452] LustreError: 166821:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788848366 with bad export cookie 17039234397923523627 [ 5201.004629] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5201.010780] LustreError: 166821:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5201.373558] Lustre: server umount lustre-MDT0001 complete [ 5211.650588] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 5214.046571] Key type lgssc unregistered [ 5214.282644] LNet: 170701:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5214.292382] LNetError: 170701:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5214.306695] LNet: Removed LNI 192.168.202.111@tcp [ 5215.091191] Key type .llcrypt unregistered [ 5215.092781] Key type ._llcrypt unregistered [ 5239.486890] Key type ._llcrypt registered [ 5239.488468] Key type .llcrypt registered [ 5239.627652] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 5240.721307] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5240.872231] alg: No test for adler32 (adler32-zlib) [ 5241.911634] Lustre: Lustre: Build Version: 2.17.57_111_g33f3974 [ 5242.124464] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 5243.823467] Key type lgssc registered [ 5245.088879] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5252.520906] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5252.549443] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5253.939184] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5258.288868] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 5264.354746] LustreError: 172115:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5264.387507] LustreError: 172115:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5267.272580] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5267.613357] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5272.176710] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 5279.209130] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5284.913708] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5291.324420] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5291.643317] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5297.463735] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 5299.858391] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5305.884465] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5312.787412] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5312.809509] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5312.819509] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5312.827281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.834928] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5313.642383] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5313.653461] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5313.678227] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5313.692333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.722271] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5314.697934] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5314.711074] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5314.717360] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5314.721450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.728989] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5315.829132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.772564] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5316.782319] Lustre: Skipped 1 previous similar message [ 5316.820709] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5316.847524] Lustre: Skipped 1 previous similar message [ 5316.858628] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5316.861886] Lustre: Skipped 1 previous similar message [ 5316.869644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.879240] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5316.882936] Lustre: Skipped 1 previous similar message [ 5317.789827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.566880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.503991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.400467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.524905] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5321.530119] Lustre: Skipped 4 previous similar messages [ 5321.547239] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5321.555281] Lustre: Skipped 4 previous similar messages [ 5321.559167] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5321.562907] Lustre: Skipped 4 previous similar messages [ 5321.566697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.574218] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5321.576969] Lustre: Skipped 4 previous similar messages [ 5322.520088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.450480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.348256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.404398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.321051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.229323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.037170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.039092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.002417] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5330.013872] Lustre: Skipped 8 previous similar messages [ 5330.045630] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5330.075449] Lustre: Skipped 8 previous similar messages [ 5330.106107] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5330.115543] Lustre: Skipped 8 previous similar messages [ 5330.124105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.141111] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5330.148561] Lustre: Skipped 8 previous similar messages [ 5330.969402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.052816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.192803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.349081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.663921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.957881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.888890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.049300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.065727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.055450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.098795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.144394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.966236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.020826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.905186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.732530] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5346.743801] Lustre: Skipped 15 previous similar messages [ 5346.758465] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5346.765974] Lustre: Skipped 15 previous similar messages [ 5346.768788] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5346.776812] Lustre: Skipped 15 previous similar messages [ 5346.783571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.797349] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5346.804932] Lustre: Skipped 15 previous similar messages [ 5347.590412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.555523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.480659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.374756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.232967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.399129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.299834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.243460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.091976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.217671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.076146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.935526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.796979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.658524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.575824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.559849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.539253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.714503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.670847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.636780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.227437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.246186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.288803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.400515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.561376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.750816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.873334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.412889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.850870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.460224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.115815] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5379.132994] Lustre: Skipped 30 previous similar messages [ 5379.156336] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5379.165483] Lustre: Skipped 30 previous similar messages [ 5379.199220] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5379.203919] Lustre: Skipped 30 previous similar messages [ 5379.210967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.233670] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5379.246767] Lustre: Skipped 30 previous similar messages [ 5380.520334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.987477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.597588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.935591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.430449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.915974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.670056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.897428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.861921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.327251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.057452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.455364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.497620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.622878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.824933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.831362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.754128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.709690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.678694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.521447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.426885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.388246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.240715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.069668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.119692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.977496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.013417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.234659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.300441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.253934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.305753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.394571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.236530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.403943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.491289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.420768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.648843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.699480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.649622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.547854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.669688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.631592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.764295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.918491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.924184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.942094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.758762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.815252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.853954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.830315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.768409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.111411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.509342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.591520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.983629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.901470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.169157] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5443.175356] Lustre: Skipped 56 previous similar messages [ 5443.196954] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5443.204370] Lustre: Skipped 56 previous similar messages [ 5443.208126] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5443.211486] Lustre: Skipped 56 previous similar messages [ 5443.214397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.182384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.195260] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5444.204057] Lustre: Skipped 57 previous similar messages [ 5445.327148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.415236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.392304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.152276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.993445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.065317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.303880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.401748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.601625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.614824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.544293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.463980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.329565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.144211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.947749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.739550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.591434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.618199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.703610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.762690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.853329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.715926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.927363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.961404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.265105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.145432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.077444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.046475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.075458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.271122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.162025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.387440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.569571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.858730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.892599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.891841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.890942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.970343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.213373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.235750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.231320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.147883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.018779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.935581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.014184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.974162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.668741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.416077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.293281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.306974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.094198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.938924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.911330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.047715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.031847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.074600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.970748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.775886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.624907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.800436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.686787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.680684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.580201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.414298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.430495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.282212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.140421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.078520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.065654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.893357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.809186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.721710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.613276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.515565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.626434] LustreError: 172134:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5517.637504] LustreError: 172134:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5517.647717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.495489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.654546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.551274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.531851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.598277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.588318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.518811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.456122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.434501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.451594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.285551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.307143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.339523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.260811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.113908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.139513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.120669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.276412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.243307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.269112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.268778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.350032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.123477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.047911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.775890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.628421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.370851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.437314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.256497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.136532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.101620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.218031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.252996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.262612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.138727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.222946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.374750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.398129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.352944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.391355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.483360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.598857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.804119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.886428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.942946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.004196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.020379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.950835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.096990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.023852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.234341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.490565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.736512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.777127] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5571.792884] Lustre: Skipped 129 previous similar messages [ 5571.832061] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5571.851149] Lustre: Skipped 129 previous similar messages [ 5571.856963] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5571.872574] Lustre: Skipped 129 previous similar messages [ 5571.882358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.010877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.029291] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5573.033954] Lustre: Skipped 129 previous similar messages [ 5573.863106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.861801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.949788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.956925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.029275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.065948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.208468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.295527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.350549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.447905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.728916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.718964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.761158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.725675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.685579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.609511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.687413] LustreError: 172134:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5590.690445] LustreError: 172134:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5590.709991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.695608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.867485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.883938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.710815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.711043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.937089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.866480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.879371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.773597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.677881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.422296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.262351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.007081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.014623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.028938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.056785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.041874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.007279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.886372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.814279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.868151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.207346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.149849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.219319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.348241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.475906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.656936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.506979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.650894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.729490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.931961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.986085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.008371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.841284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.762552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.798172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.614370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.430634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.306161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.423181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.542433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.684669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.664112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.763350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.784586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.868683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.877818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.037058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.154053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.939627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.939407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.999589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.977534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.197829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.164236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.438480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.513395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.526359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.677273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.029655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.928627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.087114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.042647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.142735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.921631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.945337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.906800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.852705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.638441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.698941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.775787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.666929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.534824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.384901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.366471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.278765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.088509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.137308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.976129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.919883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.834875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.787677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.622110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.592503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.583662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.765944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.963061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.933602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.995887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.850273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.750603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.670248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.584255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.331392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.225740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.054271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.078757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.353217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.476506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.336878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.268542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.093294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.053805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.954363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.070305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.035967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.298319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.298776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.573864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.531776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.569543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.657952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.659420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.846293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.001333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.931656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.997224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.953424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.954064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.895559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.718964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.647190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.630709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.561243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.570966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.539637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.509977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.415575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.422553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.519896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.458294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.477213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.446648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.316584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.996370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.823551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.740954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.906643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.813519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.692687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.552151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.493868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.429321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.475656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.333246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.242448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.096485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.028548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.931079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.886574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.680614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.522562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.196539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.060975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.872970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.601199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.272519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.910113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.631653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.434926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.094923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.940353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.693699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.609189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.294769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.185344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.986637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.958345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.754386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.681618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.481358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.513991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.455683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.232256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.880529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.716459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.485553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.286309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.928755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.720077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.415452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.329376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.049451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.842380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.632943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.443837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.213666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.047175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.879668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.814421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.544736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.394855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.298057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.398438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.296541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.231488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.222741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.353153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.019468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.001721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.910196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.885662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.788783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.005102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.000274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.910206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.750975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.936817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.938869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.013853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.927830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.878700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.939781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.039072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.046268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.990975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.866167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.739503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.666491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.633059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.422095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.322329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.344613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.153910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.136126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.062343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.050522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.750722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.552996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.340380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.183455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.235756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.213416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.368557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.288667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.283891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.162493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.234909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.292237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.275126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.202188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.934276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.857520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.662996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.617627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.437735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.322387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.241777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.231311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.037788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.794586] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5827.800627] Lustre: Skipped 268 previous similar messages [ 5827.837278] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5827.853880] Lustre: Skipped 268 previous similar messages [ 5827.861500] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5827.870681] Lustre: Skipped 268 previous similar messages [ 5827.875559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.686725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.459765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.467598] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5829.480755] Lustre: Skipped 269 previous similar messages [ 5830.326293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.061637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.694940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.440637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.213296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.106608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.934205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.845800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.560437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.418805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.153603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.976889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.850805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.793295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.657657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.629780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.424883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.245139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.008724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.876480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.740478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.637704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.676639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.607137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.508414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.153182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.956916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.647875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.368181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.182885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.006215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.868114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.663963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.386876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.196457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.142108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.193349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.187648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.238775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.146827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.183093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.029305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.973937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.780899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.810657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.737103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.688177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.474309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.395544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.258279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.971503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.619876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.620586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.333987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.128954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.897762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.725639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.559381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.446725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.274773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.274756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.136621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.872391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.621556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.469723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.369122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.336993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.212883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.161370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.944971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.789228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.648332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.484958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.309293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.266626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.217547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.150333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.999274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.001374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.807582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.637814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.379324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.341726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.191095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.053646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.035523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.053721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.922693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.783700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.460874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.219175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.958384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.841116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.601753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.422347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.132852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.935378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.779839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.663102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.507314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.383686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.231889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.265277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.217632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.042485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.979732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.990251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.856936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.922276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.854250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.773877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.649624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.522746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.350720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.198418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.837056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.749972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.649352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.696266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.672294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.635901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.525821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.523960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.355930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.184497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.943607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.706721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.591465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.757778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.762572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.756869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.883682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.705588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.533548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.441507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.416962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.694989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.584828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.517756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.267577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.233947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.025498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.819127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.715126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.787861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.785254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.793818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.649813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.563466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.454525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.311801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.285933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.218145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.207873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.184415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.033195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.034800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.100255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.044743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.008906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.003321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.015800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.154725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.989849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.989034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.961460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.072603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.063943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.120546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.938082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.090414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.021739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.073294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.110287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.133996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.144111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.331103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.538642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.538518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.282875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.327763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.080403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.047104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.979686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.291467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.475884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.621493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.791898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.749742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.179799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.319598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.670644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.631565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.402359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.321970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.240302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.128571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.937688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.887043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.987753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.023153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.826984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.745936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.760299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.732421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.625217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.589630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.384929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.241022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.102972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.978839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.831221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.810331] LustreError: 172134:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6022.821608] LustreError: 172134:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6022.841357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.884933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.949526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.037111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.893737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.103689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.004213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.993663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.838540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.838581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.701092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.838919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.871950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.168282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.502964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.469866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.574699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.534387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.827852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.643309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.635481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.816193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.716483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.723188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.752383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.773763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.668229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.637982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.696598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.607077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.627240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.558901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.706170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.632173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.715068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.711137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.997460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.285153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.219870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.287179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.507979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.600376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.736198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.751724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.778842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.714347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.571086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.287076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.354395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.360277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.343946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.164817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.942096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.638079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.442594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.215417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.030769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.755462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.597903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.497426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.608589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.797734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.832896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.758597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.660172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.508084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.306859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.251456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.955394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.012736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.824960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.766818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.452663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.396261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.202599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.092993] LustreError: 172134:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6096.104152] LustreError: 172134:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6096.118400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.029736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.169809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.058717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.873298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.605821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.444592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.268217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.152501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.062249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.784629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.645786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.611678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.475258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.346518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.224857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.982158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.729991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.491357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.261048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.145726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.891813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.665939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.504579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.239238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.893889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.763519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.598342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.409445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.282443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.086682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.927787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.815124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.549811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.387329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.198277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.912072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.738035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.780208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.569925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.469580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.215079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.047088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.810732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.844239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.885606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.936475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.562923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.303541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.955037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.698628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.538444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.475133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.372375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.618216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.703400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.714340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.504507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.350754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.165629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.038210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.742596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.463018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.128021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.032389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.756502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.630465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.455446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.349733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.209958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.986301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.847217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.634051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.253278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.103325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.862270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.594174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.464141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.191696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.027969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.743564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.524508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.304971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.131975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.856865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.677079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.446832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.293676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.964500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.701827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.470858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.195578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.985883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.765817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.406514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.299461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.028513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.796556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.520496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.328975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.033897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.867956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.514557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.355347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.058440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.754542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.359308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.069860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.737303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.530052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.115522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.833821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.482417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.182622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.913685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.579164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.307523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.015072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.831511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.597389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.289592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.057961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.819288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.507703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.275949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.012676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.630474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.239984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.815961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.399285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.071262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.670960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.446959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.079557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.817566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.559543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.436744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.084301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.767090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.422705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.125606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.798484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.501818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.285155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.156343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.960656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.707023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.426025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.153687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.924825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.609880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.304830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.126315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.002772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.711105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.313448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.981838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.818551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.707389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.563401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.322203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.085649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.762737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.533218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.452473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.233444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.850823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.579782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.223840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.929118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.561635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.382622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.159293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.890129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.743946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.672377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.542577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.393766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.995852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.724962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.400332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.134257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.818861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.570590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.195539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.951963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.738912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.460606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.156196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6547.139532] LustreError: 173935:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6547.146812] LustreError: 172134:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6904.802131] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6904.807530] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6904.814021] Lustre: Skipped 478 previous similar messages [ 6904.816692] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6905.824203] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6905.835805] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6907.873034] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6907.878378] Lustre: Skipped 1 previous similar message [ 6910.542832] Lustre: server umount lustre-OST0000 complete [ 6916.067376] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6916.073582] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6920.596462] Lustre: server umount lustre-MDT0000 complete [ 6923.241655] LustreError: 172115:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6923.263401] LustreError: 172115:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6924.114477] LustreError: 209538:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788850089 with bad export cookie 13255447640234091510 [ 6924.116312] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6924.126986] LustreError: 209538:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6924.377848] Lustre: server umount lustre-MDT0001 complete [ 6933.047227] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 6935.247808] Key type lgssc unregistered [ 6935.473686] LNet: 227298:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6935.478838] LNetError: 227298:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6935.497713] LNet: Removed LNI 192.168.202.111@tcp [ 6936.061234] Key type .llcrypt unregistered [ 6936.063370] Key type ._llcrypt unregistered [ 6953.725554] Key type ._llcrypt registered [ 6953.727156] Key type .llcrypt registered [ 6953.843855] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 6954.427986] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6954.599512] alg: No test for adler32 (adler32-zlib) [ 6955.625497] Lustre: Lustre: Build Version: 2.17.57_111_g33f3974 [ 6955.791448] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 6957.431278] Key type lgssc registered [ 6958.307482] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6963.482725] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6963.510358] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6964.810453] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6968.553424] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 6973.923308] LustreError: 228688:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6973.941957] LustreError: 228688:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 6976.501432] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6976.787684] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6980.533488] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 6986.537892] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6991.071250] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6997.156159] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6997.510271] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7002.919545] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7009.145262] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7009.865117] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7014.315917] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7015.765880] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7015.774396] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7015.782515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7015.805799] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7015.806288] LustreError: 228708:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7021.030566] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7021.047747] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7022.272158] Lustre: lustre-OST0000: Not available for connect from 192.168.202.11@tcp (stopping) [ 7022.281258] Lustre: Skipped 1 previous similar message [ 7023.452054] Lustre: server umount lustre-OST0000 complete [ 7035.847125] LustreError: 228708:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7055.531565] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7055.758643] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7060.330569] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7060.904129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7060.918586] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7060.918695] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7060.940660] Lustre: Skipped 1 previous similar message [ 7065.531980] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7066.565990] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7070.244996] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff934f84593800.ost_server_uuid 50 [ 7071.194857] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff934f84593800.ost_server_uuid in FULL state after 0 sec [ 7076.333544] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7076.336113] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7076.353206] Lustre: Skipped 1 previous similar message [ 7076.354824] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7076.370431] LustreError: Skipped 1 previous similar message [ 7079.844585] Lustre: server umount lustre-OST0000 complete [ 7084.512161] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7084.515539] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7084.524428] Lustre: Skipped 1 previous similar message [ 7084.534128] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7084.538454] Lustre: Skipped 1 previous similar message [ 7088.892697] Lustre: server umount lustre-MDT0000 complete [ 7091.682166] LustreError: 228689:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7091.703391] LustreError: 228689:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7091.833828] LustreError: 229425:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788850256 with bad export cookie 10593337604468428922 [ 7091.834327] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7091.840638] LustreError: 229425:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7092.072258] Lustre: server umount lustre-MDT0001 complete [ 7100.985865] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 7103.405637] Key type lgssc unregistered [ 7103.636329] LNet: 233895:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7103.642324] LNetError: 233895:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7104.693061] LNet: Removed LNI 192.168.202.111@tcp [ 7105.438655] Key type .llcrypt unregistered [ 7105.440114] Key type ._llcrypt unregistered [ 7129.984493] Key type ._llcrypt registered [ 7129.986814] Key type .llcrypt registered [ 7130.062932] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 7130.788331] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7130.900788] alg: No test for adler32 (adler32-zlib) [ 7131.856348] Lustre: Lustre: Build Version: 2.17.57_111_g33f3974 [ 7132.031481] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 7133.663277] Key type lgssc registered [ 7134.480600] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7141.771172] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7141.781417] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7143.203586] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7146.379927] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7153.346653] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7153.593607] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7156.781219] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7159.357813] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7164.365443] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7164.636623] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7169.782550] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7170.118812] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7176.365427] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7176.503198] Lustre: lustre-OST0001: new disk, initializing [ 7176.506219] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7176.511525] Lustre: 237903:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7176.579871] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7181.302300] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7185.977326] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7185.987176] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7186.082336] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7190.893167] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7194.105472] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7206.883257] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7206.888380] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7206.896269] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [ 7206.909267] Lustre: Skipped 2 previous similar messages [ 7206.910856] Lustre: Skipped 1 previous similar message [ 7207.076222] Lustre: server umount lustre-MDT0000 complete [ 7209.889513] LustreError: 238795:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788850375 with bad export cookie 2403337254647870606 [ 7209.896829] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7209.898111] LustreError: 236337:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200001b72:0x1:0x0]: rc = -5 [ 7209.900092] LustreError: 238795:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7210.099865] Lustre: server umount lustre-MDT0001 complete [ 7223.196100] Lustre: server umount lustre-OST0000 complete [ 7236.423879] Lustre: server umount lustre-OST0001 complete [ 7251.503791] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 7258.230420] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7258.512365] LustreError: 241014:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7258.534491] LustreError: 241014:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7258.582867] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7261.660781] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7263.723754] LustreError: 241015:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7268.234550] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7268.517209] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7271.770775] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7273.617398] Lustre: 242156:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7278.748904] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7283.125295] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7284.137755] LustreError: 242512:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7284.152087] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 7288.722474] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7288.911573] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7288.916254] Lustre: Skipped 1 previous similar message [ 7292.486378] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7293.049667] Lustre: mylustre-OST0001: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:33) [ 7293.052202] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:129) [ 7297.980831] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7300.889514] Lustre: 244027:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7309.185663] Lustre: server umount mylustre-MDT0000 complete [ 7309.279958] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7309.289925] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7309.298537] Lustre: Skipped 1 previous similar message [ 7309.304070] LustreError: 241765:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7312.117259] LustreError: 240989:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788850477 with bad export cookie 2403337254647873469 [ 7312.125272] LustreError: 240989:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7312.129162] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7313.893291] Lustre: mylustre-MDT0000-lwp-OST0000: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7313.900061] Lustre: mylustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7313.901051] LustreError: 241011:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7313.901519] Lustre: Skipped 3 previous similar messages [ 7313.922964] Lustre: Skipped 3 previous similar messages [ 7317.922597] Lustre: mylustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7318.405352] Lustre: server umount mylustre-MDT0001 complete [ 7321.337493] Lustre: server umount mylustre-OST0000 complete [ 7324.430568] Lustre: server umount mylustre-OST0001 complete [ 7339.380728] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 7346.482581] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7346.861680] LustreError: 246334:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7346.877752] LustreError: 246334:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 7346.917850] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7349.847110] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7355.772103] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7359.383752] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7361.471604] Lustre: 247476:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7366.019081] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7366.180588] LustreError: 247831:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7366.190206] LustreError: 247831:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7370.809205] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7371.773606] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 7372.787512] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 7377.033655] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7381.362417] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7382.529055] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:65) [ 7386.966951] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7389.534427] Lustre: 249344:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7396.833227] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7396.846164] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7396.854558] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7396.857159] Lustre: Skipped 1 previous similar message [ 7399.735509] Lustre: server umount tfs-MDT0000 complete [ 7401.553230] LustreError: 246311:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788850566 with bad export cookie 2403337254647875646 [ 7401.555775] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7401.563143] LustreError: 246311:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7401.788039] Lustre: server umount tfs-MDT0001 complete [ 7414.265989] Lustre: server umount tfs-OST0000 complete [ 7426.249322] Lustre: server umount tfs-OST0001 complete [ 7439.870359] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 7447.170673] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7447.544862] LustreError: 251593:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7447.552578] LustreError: 251593:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 7447.596281] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7447.605573] Lustre: Skipped 3 previous similar messages [ 7450.458380] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7456.655703] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7460.184462] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7462.340523] Lustre: 252733:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7466.982459] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7471.358683] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7473.394742] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 7477.080857] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7481.287794] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7481.348791] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:97) [ 7481.348962] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 7486.528505] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7489.475908] Lustre: 254601:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7498.209069] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 02:57:41 (1788850661) [ 7517.173336] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7517.189071] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7517.199918] Lustre: Skipped 5 previous similar messages [ 7517.214708] Lustre: Skipped 7 previous similar messages [ 7522.289061] LustreError: 251594:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7522.307054] LustreError: 251594:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 7522.415295] Lustre: server umount lustre-MDT0000 complete [ 7528.593588] LustreError: 254604:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788850693 with bad export cookie 2403337254647877564 [ 7528.612265] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7528.622114] LustreError: 254604:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7532.515561] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7532.527157] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7532.529707] Lustre: Skipped 2 previous similar messages [ 7532.555550] Lustre: Skipped 1 previous similar message [ 7535.394088] Lustre: server umount lustre-MDT0001 complete [ 7546.821150] Lustre: server umount lustre-OST0000 complete [ 7557.521868] Lustre: server umount lustre-OST0001 complete [ 7565.057158] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 7574.243716] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 7607.957100] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7608.194538] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7608.230422] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7608.336444] Lustre: lustre-MDT0000: new disk, initializing [ 7608.479079] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 7608.483901] Lustre: Skipped 3 previous similar messages [ 7608.494941] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7612.380572] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7621.595991] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7621.747212] Lustre: 257793:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7621.815610] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7621.821707] Lustre: Skipped 1 previous similar message [ 7621.885922] Lustre: lustre-MDT0001: new disk, initializing [ 7622.018453] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7622.037634] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7626.353215] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7635.292898] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7635.520948] Lustre: lustre-OST0000: new disk, initializing [ 7635.524832] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7635.534969] Lustre: 259377:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7636.690964] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7636.710138] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7636.862109] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7641.498295] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7653.365938] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7653.510385] Lustre: lustre-OST0001: new disk, initializing [ 7653.514127] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7653.521631] Lustre: 260254:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7654.957134] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7654.964141] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7655.091895] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7659.696492] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7671.033535] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 03:00:35 (1788850835) [ 7686.202686] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 03:00:50 (1788850850) [ 7703.520984] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7703.521449] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7703.536067] LustreError: Skipped 1 previous similar message [ 7703.562513] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7703.571174] Lustre: Skipped 3 previous similar messages [ 7707.948973] Lustre: server umount lustre-OST0000 complete [ 7711.721845] LustreError: 259363:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7711.742503] LustreError: 259363:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 8 previous similar messages [ 7717.710956] Lustre: server umount lustre-MDT0000 complete [ 7722.399138] LustreError: 260253:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788850887 with bad export cookie 2403337254647879335 [ 7722.402396] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7722.409377] LustreError: 260253:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7728.921876] Lustre: server umount lustre-MDT0001 complete [ 7744.651077] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 7820.516795] Lustre: server umount lustre-OST0001 complete [ 7827.003623] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 7840.263310] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 7875.606694] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7875.959823] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7876.005422] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7876.130627] Lustre: lustre-MDT0000: new disk, initializing [ 7876.217973] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7876.226773] Lustre: Skipped 3 previous similar messages [ 7876.245661] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7881.267577] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7891.789562] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7891.979752] Lustre: 265115:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7892.131535] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7892.138774] Lustre: Skipped 1 previous similar message [ 7892.286779] Lustre: lustre-MDT0001: new disk, initializing [ 7892.576326] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7892.586351] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7897.520817] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7907.785976] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7913.349659] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7920.897293] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7921.133980] Lustre: 267084:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7922.744856] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7922.899647] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7927.976865] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7938.256633] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7948.768595] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7948.772279] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7948.774135] LustreError: Skipped 1 previous similar message [ 7948.774916] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7948.774920] Lustre: Skipped 9 previous similar messages [ 7948.790248] Lustre: Skipped 6 previous similar messages [ 7950.796860] Lustre: server umount lustre-OST0000 complete [ 7959.013114] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7961.135065] Lustre: server umount lustre-MDT0000 complete [ 7964.521519] LustreError: 265919:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788851129 with bad export cookie 2403337254647880812 [ 7964.536092] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7964.813591] Lustre: server umount lustre-MDT0001 complete [ 7978.169898] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 7981.755477] Key type lgssc unregistered [ 7982.219352] LNet: 268842:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7982.229172] LNetError: 268842:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7982.244775] LNet: Removed LNI 192.168.202.111@tcp [ 7983.460170] Key type .llcrypt unregistered [ 7983.461868] Key type ._llcrypt unregistered [ 8019.759454] Key type ._llcrypt registered [ 8019.760965] Key type .llcrypt registered [ 8019.952351] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 8021.220203] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8021.300190] alg: No test for adler32 (adler32-zlib) [ 8022.470966] Lustre: Lustre: Build Version: 2.17.57_111_g33f3974 [ 8022.762846] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 8024.449786] Key type lgssc registered [ 8026.206356] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8035.762237] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8035.798229] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8037.274956] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8042.244615] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8047.590915] LustreError: 270308:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8047.619124] LustreError: 270308:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8051.010696] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8051.210843] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8055.861201] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8063.046467] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8068.224270] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8075.878164] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8076.334991] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8082.554807] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8083.437632] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8092.007613] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8093.188861] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8098.290490] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8098.301537] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8098.332742] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8101.364695] Lustre: server umount lustre-OST0000 complete [ 8107.492425] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8107.505812] LustreError: Skipped 1 previous similar message [ 8107.514809] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8107.533996] Lustre: Skipped 1 previous similar message [ 8107.539660] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8107.555588] Lustre: Skipped 1 previous similar message [ 8108.514098] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8113.632197] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8113.650570] Lustre: Skipped 3 previous similar messages [ 8118.753344] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8118.771235] Lustre: Skipped 1 previous similar message [ 8120.287337] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8120.543827] Lustre: server umount lustre-MDT0000 complete [ 8123.874804] LustreError: 271036:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8124.663986] LustreError: 270289:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788851289 with bad export cookie 12028756559978601278 [ 8124.665434] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8124.671995] LustreError: 270289:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8124.927266] Lustre: server umount lustre-MDT0001 complete [ 8137.223591] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 8140.406235] Key type lgssc unregistered [ 8140.704426] LNet: 273897:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8140.710132] LNetError: 273897:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8140.734990] LNet: Removed LNI 192.168.202.111@tcp [ 8141.443470] Key type .llcrypt unregistered [ 8141.446359] Key type ._llcrypt unregistered [ 8159.226768] Key type ._llcrypt registered [ 8159.228543] Key type .llcrypt registered [ 8159.381196] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 8160.516626] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8160.628897] alg: No test for adler32 (adler32-zlib) [ 8161.686333] Lustre: Lustre: Build Version: 2.17.57_111_g33f3974 [ 8161.867416] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 8163.508077] Key type lgssc registered [ 8164.521832] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8173.482452] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8173.501857] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8189.023505] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8194.143879] LustreError: 275085:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.111@tcp: failed processing log, type 4: rc = -110 [ 8220.895389] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8228.927580] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8237.583640] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8239.939473] Lustre: server umount lustre-OST0000 complete [ 8257.550939] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 8260.520403] Key type lgssc unregistered [ 8260.877262] LNet: 276716:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8260.883803] LNetError: 276716:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8260.899932] LNet: Removed LNI 192.168.202.111@tcp [ 8261.812677] Key type .llcrypt unregistered [ 8261.814759] Key type ._llcrypt unregistered [ 8305.243512] Key type ._llcrypt registered [ 8305.245665] Key type .llcrypt registered [ 8305.382090] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 8306.704935] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8306.743418] alg: No test for adler32 (adler32-zlib) [ 8307.907871] Lustre: Lustre: Build Version: 2.17.57_111_g33f3974 [ 8308.122444] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 8309.760288] Key type lgssc registered [ 8311.150564] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8326.962095] loop0: detected capacity change from 0 to 204800000 [ 8327.194472] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8327.203821] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8327.741578] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8329.270960] loop0: detected capacity change from 0 to 204800000 [ 8329.455829] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8329.469556] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8329.981508] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8331.079870] loop0: detected capacity change from 0 to 204800000 [ 8331.279777] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8331.289817] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8331.370850] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8332.694753] loop0: detected capacity change from 0 to 204800000 [ 8332.971493] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8332.990271] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8333.194692] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8334.260844] loop0: detected capacity change from 0 to 204800000 [ 8334.288790] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8338.495845] loop0: detected capacity change from 0 to 204800000 [ 8338.523408] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8343.413501] loop0: detected capacity change from 0 to 204800000 [ 8343.486371] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8348.233648] loop0: detected capacity change from 0 to 204800000 [ 8348.264780] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8353.389462] loop0: detected capacity change from 0 to 204800000 [ 8353.505268] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8353.569633] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8353.625103] Lustre: 280131:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8353.640178] Lustre: 280131:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8353.651550] Lustre: 280131:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8353.660869] Lustre: 280131:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8353.672624] Lustre: 280131:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8353.682142] Lustre: 280131:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8353.691228] Lustre: 280131:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8353.699645] Lustre: 280131:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8353.707227] Lustre: 280131:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8353.719417] Lustre: 280131:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8353.727455] Lustre: 280131:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8353.736560] Lustre: 280131:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8353.747573] Lustre: 280131:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8353.756652] Lustre: 280131:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8360.024057] Lustre: server umount MGS complete [ 8360.949528] loop0: detected capacity change from 0 to 204800000 [ 8361.037760] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8361.747718] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8361.821672] Lustre: lustre-MDT0000: Aborting client recovery [ 8362.692236] loop1: detected capacity change from 0 to 204800000 [ 8362.801636] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8362.916770] Lustre: 280567:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8362.933917] Lustre: 280567:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8362.950718] Lustre: 280567:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8362.963534] Lustre: 280567:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8362.974419] Lustre: 280567:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8362.982497] Lustre: 280567:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8362.990437] Lustre: 280567:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8363.004987] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8363.159576] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8363.234025] Lustre: lustre-MDT0001: Aborting client recovery [ 8364.143550] loop2: detected capacity change from 0 to 204800000 [ 8364.274236] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8364.368883] Lustre: 280647:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8364.383911] Lustre: 280647:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8364.401331] Lustre: 280647:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8364.419869] Lustre: 280647:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8364.429971] Lustre: 280647:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8364.444551] Lustre: 280647:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8364.453206] Lustre: 280647:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8364.552086] Lustre: Found index 0 for lustre-OST0000, updating log [ 8364.583778] Lustre: 280666:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8364.647136] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8364.672239] Lustre: lustre-OST0000: Aborting recovery [ 8365.505621] loop3: detected capacity change from 0 to 204800000 [ 8365.646430] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8365.780629] Lustre: 280731:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8365.808784] Lustre: 280731:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8365.827403] Lustre: 280731:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8365.844464] Lustre: 280731:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8365.861469] Lustre: 280731:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8365.876367] Lustre: 280731:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8365.886684] Lustre: 280731:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8365.918544] Lustre: Found index 1 for lustre-OST0001, updating log [ 8365.957484] Lustre: 280736:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8366.085770] Lustre: lustre-OST0001: Aborting recovery [ 8371.190276] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8371.231500] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8371.250587] LustreError: 280511:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 8371.271485] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8371.299392] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8373.222762] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8373.232391] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8375.284097] LustreError: 280476:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1228us: rc = -2 [ 8375.791424] LustreError: 280473:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 10543us: rc = -2 [ 8375.816146] LustreError: 280473:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 20 previous similar messages [ 8389.093121] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8389.116442] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8389.138438] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8393.713725] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8393.726685] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8393.764665] Lustre: Skipped 2 previous similar messages [ 8395.349506] Lustre: server umount lustre-MDT0000 complete [ 8396.753740] LustreError: 280458:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788851561 with bad export cookie 13833500274612104040 [ 8396.754239] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8396.766110] LustreError: 280458:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8398.817282] LustreError: 280474:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8398.817699] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8398.840649] LustreError: 280474:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8398.862834] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8398.865754] Lustre: Skipped 3 previous similar messages [ 8401.824345] LustreError: 280474:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8401.824550] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8401.838781] LustreError: 280474:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8401.868049] Lustre: Skipped 1 previous similar message [ 8405.939251] LustreError: 280475:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8405.939508] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8405.970833] LustreError: 280475:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8405.976618] Lustre: Skipped 1 previous similar message [ 8411.103131] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8411.104135] LustreError: 280473:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8411.158783] LustreError: 280473:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8411.474596] Lustre: server umount lustre-MDT0001 complete [ 8419.225302] Lustre: server umount lustre-OST0000 complete [ 8426.616437] Lustre: server umount lustre-OST0001 complete [ 8436.336693] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 03:13:20 (1788851600) [ 8472.795996] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 8485.869868] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 8531.755413] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8532.181108] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8532.234605] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8532.421858] Lustre: lustre-MDT0000: new disk, initializing [ 8532.489333] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8532.500144] Lustre: Skipped 1 previous similar message [ 8532.521550] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8538.147882] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8551.697875] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8551.885395] Lustre: 284823:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8551.999940] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8552.006160] Lustre: Skipped 1 previous similar message [ 8552.109799] Lustre: lustre-MDT0001: new disk, initializing [ 8552.286098] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8552.322213] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8552.328754] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8557.780279] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8567.829539] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8573.486602] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8580.326187] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8580.629427] Lustre: lustre-OST0000: new disk, initializing [ 8580.633555] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8580.648558] Lustre: 286794:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8580.782346] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8587.854529] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8589.363884] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8589.376188] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8589.534784] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8597.901035] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8604.331274] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8616.225199] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8624.658150] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8671.714588] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8671.728113] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8671.735233] LustreError: Skipped 1 previous similar message [ 8671.738124] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8671.758261] Lustre: Skipped 2 previous similar messages [ 8671.785985] Lustre: Skipped 3 previous similar messages [ 8677.847918] Lustre: server umount lustre-OST0000 complete [ 8681.955710] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8681.968747] Lustre: Skipped 1 previous similar message [ 8692.192882] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8692.213551] Lustre: Skipped 7 previous similar messages [ 8696.288020] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8696.792129] Lustre: server umount lustre-MDT0000 complete [ 8697.320948] LustreError: 286007:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8697.336410] LustreError: 286007:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8700.918054] LustreError: 285620:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788851866 with bad export cookie 13833500274612106742 [ 8700.921042] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8700.935532] LustreError: 285620:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8701.258704] Lustre: server umount lustre-MDT0001 complete [ 8712.865589] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8717.639244] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8723.897610] Lustre: server umount MGS complete [ 8736.453593] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8736.957727] LustreError: 290713:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8736.996618] LustreError: 290726:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8737.082957] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8741.855253] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8752.157800] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8758.742086] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8766.170021] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8772.744666] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8782.360327] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8782.826877] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8782.842435] Lustre: Skipped 1 previous similar message [ 8789.789583] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8798.205115] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8843.236939] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8843.245780] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8843.249534] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8843.261692] Lustre: Skipped 1 previous similar message [ 8843.276022] Lustre: Skipped 1 previous similar message [ 8845.756402] Lustre: server umount lustre-OST0000 complete [ 8849.889657] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8849.899905] LustreError: Skipped 1 previous similar message [ 8855.996653] Lustre: server umount lustre-MDT0000 complete [ 8858.593722] LustreError: 291883:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8858.608318] LustreError: 291883:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 8861.063764] LustreError: 290693:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788852026 with bad export cookie 13833500274612108583 [ 8861.068219] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8861.086449] LustreError: 290693:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8861.647813] Lustre: server umount lustre-MDT0001 complete [ 8875.512613] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 8879.722965] Key type lgssc unregistered [ 8880.097283] LNet: 294676:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8880.102838] LNetError: 294676:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8880.117784] LNet: Removed LNI 192.168.202.111@tcp [ 8881.168582] Key type .llcrypt unregistered [ 8881.170237] Key type ._llcrypt unregistered [ 8928.318710] Key type ._llcrypt registered [ 8928.321224] Key type .llcrypt registered [ 8928.426347] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 8952.763782] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 8953.552428] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8953.619867] alg: No test for adler32 (adler32-zlib) [ 8954.960494] Lustre: Lustre: Build Version: 2.17.57_111_g33f3974 [ 8955.288568] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 8957.171346] Key type lgssc registered [ 8958.745669] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9002.595535] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9002.613919] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9003.760219] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9003.785289] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9003.834347] Lustre: lustre-MDT0000: new disk, initializing [ 9003.903742] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9003.919450] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9009.877377] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9023.789263] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9023.941788] Lustre: 298635:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9024.000465] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9024.003944] Lustre: Skipped 1 previous similar message [ 9024.102158] Lustre: lustre-MDT0001: new disk, initializing [ 9024.246548] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9024.317216] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9024.330093] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9028.860322] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9038.240190] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9044.412389] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9052.130705] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9052.483725] Lustre: lustre-OST0000: new disk, initializing [ 9052.487605] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9052.499669] Lustre: 300603:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9052.636820] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9059.449981] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9060.941034] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9060.953825] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9061.122219] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9069.210898] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9077.209181] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9089.877758] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9094.281785] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9103.205914] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9153.510023] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9153.512360] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9153.565213] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9153.573788] Lustre: Skipped 1 previous similar message [ 9156.830359] Lustre: server umount lustre-OST0000 complete [ 9162.721444] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9162.731583] LustreError: Skipped 1 previous similar message [ 9162.737086] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9162.747451] Lustre: Skipped 1 previous similar message [ 9162.753294] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9163.752226] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9163.768548] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9163.777664] Lustre: Skipped 1 previous similar message [ 9169.704666] Lustre: server umount lustre-MDT0000 complete [ 9173.831921] LustreError: 298629:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788852338 with bad export cookie 8753338394405544656 [ 9173.847884] LustreError: 298629:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9173.852980] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9174.162599] Lustre: server umount lustre-MDT0001 complete [ 9184.959270] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9190.151412] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9195.728458] Lustre: server umount MGS complete [ 9210.111906] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9210.624080] LustreError: 304430:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9210.633640] LustreError: 304430:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 9210.648386] LustreError: 304443:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9210.716640] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9215.972578] LustreError: 304431:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9216.558559] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9220.072067] LustreError: 304430:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9225.186572] LustreError: 304431:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9227.618918] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9227.929495] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9233.069679] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9240.629463] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9246.056976] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9253.552102] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9253.837149] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9259.342420] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9267.536060] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9296.879183] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9296.887242] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9296.905047] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9301.798151] Lustre: server umount lustre-OST0000 complete [ 9308.129993] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9308.132481] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9308.142503] Lustre: Skipped 2 previous similar messages [ 9308.145151] Lustre: Skipped 2 previous similar messages [ 9314.600610] Lustre: server umount lustre-MDT0000 complete [ 9318.369175] LustreError: 304431:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9318.390742] LustreError: 304431:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9319.775331] LustreError: 304412:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788852484 with bad export cookie 8753338394405546595 [ 9319.779329] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9319.791254] LustreError: 304412:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9320.262127] Lustre: server umount lustre-MDT0001 complete [ 9333.022595] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 9336.285932] Key type lgssc unregistered [ 9336.562781] LNet: 308376:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9336.568119] LNetError: 308376:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9336.578143] LNet: Removed LNI 192.168.202.111@tcp [ 9337.603520] Key type .llcrypt unregistered [ 9337.605848] Key type ._llcrypt unregistered [ 9363.541050] Key type ._llcrypt registered [ 9363.543343] Key type .llcrypt registered [ 9363.801217] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 9385.725757] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 9387.281204] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9387.424222] alg: No test for adler32 (adler32-zlib) [ 9388.735598] Lustre: Lustre: Build Version: 2.17.57_111_g33f3974 [ 9389.029321] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 9390.751319] Key type lgssc registered [ 9391.991095] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9418.971878] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 9436.052488] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9436.090071] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9437.313736] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9437.335576] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9437.406277] Lustre: lustre-MDT0000: new disk, initializing [ 9437.489309] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9437.506610] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9443.162907] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9454.243197] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9454.560197] Lustre: lustre-OST0000: new disk, initializing [ 9454.567737] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9454.571583] Lustre: Skipped 1 previous similar message [ 9454.575804] Lustre: 312429:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9454.677748] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9460.868427] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9460.879201] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9460.906933] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9461.961889] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10351.586232] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10351.615166] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10355.524797] Lustre: server umount lustre-MDT0000 complete [10359.501618] LustreError: 311616:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788853524 with bad export cookie 15496808721008190019 [10359.509516] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10359.510020] LustreError: 311616:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10359.615796] Lustre: server umount lustre-OST0000 complete [10387.159694] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [10397.720987] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [10444.780468] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10445.048085] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10445.074061] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10445.136889] Lustre: lustre-MDT0000: new disk, initializing [10445.308960] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10445.355283] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10452.056621] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10464.081313] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10464.363771] Lustre: 316774:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10464.502118] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10464.507247] Lustre: Skipped 1 previous similar message [10464.633382] Lustre: lustre-MDT0001: new disk, initializing [10464.728750] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10464.778622] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10464.785894] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10470.718299] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10481.054871] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10486.099714] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10492.350052] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10492.927674] Lustre: lustre-OST0000: new disk, initializing [10492.935298] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10492.945988] Lustre: 318744:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10493.095260] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10499.585886] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10500.707340] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10500.723157] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10501.026787] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10510.484133] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10516.329056] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10516.658744] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10522.447331] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10522.616423] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10526.704452] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10526.704496] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10526.726296] LustreError: Skipped 1 previous similar message [10526.733854] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10526.786303] Lustre: Skipped 1 previous similar message [10530.500866] Lustre: server umount lustre-OST0000 complete [10536.418298] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10536.430367] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10536.449218] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10536.457783] Lustre: Skipped 1 previous similar message [10540.436172] Lustre: server umount lustre-MDT0000 complete [10545.066693] LustreError: 316766:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788853710 with bad export cookie 15496808721011130838 [10545.069338] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10545.095974] LustreError: 316766:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10545.688036] Lustre: server umount lustre-MDT0001 complete [10550.550376] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10553.046570] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 03:48:36 (1788853716) [10560.664686] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10561.741931] LustreError: 321056:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10561.773449] LustreError: 321056:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10561.837113] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10567.148095] LustreError: 321057:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10567.566527] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10571.236052] LustreError: 321056:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10576.354667] LustreError: 321057:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10580.546253] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10581.248797] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10588.885771] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10598.090606] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10603.739684] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10609.809788] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10610.214853] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10618.759684] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10627.540583] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10635.508238] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10635.753186] Lustre: lustre-OST0001: new disk, initializing [10635.756591] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10635.762324] Lustre: 323893:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10635.915218] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10643.561383] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10644.108548] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10644.129717] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10644.191545] LustreError: 324271:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10645.309017] LustreError: 324271:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10645.328947] LustreError: 324271:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10646.373389] LustreError: 324271:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10646.388691] LustreError: 324271:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10648.502996] LustreError: 324271:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10648.517292] LustreError: 324271:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 4 previous similar messages [10652.717475] LustreError: 324271:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10652.738421] LustreError: 324271:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 6 previous similar messages [10656.518635] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10656.813780] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10660.966453] LustreError: 324279:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10660.986407] LustreError: 324279:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [10662.759596] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff934f8728a800.ost_server_uuid 50 [10664.257137] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff934f8728a800.ost_server_uuid in FULL state after 0 sec [10670.740602] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff934f8728a800.ost_server_uuid 50 [10672.532324] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff934f8728a800.ost_server_uuid in IDLE state after 0 sec [10679.786896] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10690.017531] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10690.019383] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10690.022109] LustreError: Skipped 1 previous similar message [10690.024148] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10690.024156] Lustre: Skipped 1 previous similar message [10690.055067] Lustre: Skipped 2 previous similar messages [10695.660508] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10695.669383] Lustre: Skipped 3 previous similar messages [10703.328496] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10703.569786] Lustre: server umount lustre-OST0001 complete [10705.894816] LustreError: 322878:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10705.920929] LustreError: 322878:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10711.014954] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10711.023691] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10711.029462] Lustre: Skipped 1 previous similar message [10711.038891] Lustre: Skipped 5 previous similar messages [10716.138400] LustreError: 322876:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10716.164510] LustreError: 322876:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [10716.522514] Lustre: server umount lustre-OST0000 complete [10721.251450] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10721.273145] Lustre: Skipped 1 previous similar message [10726.934808] Lustre: server umount lustre-MDT0000 complete [10731.494153] LustreError: 321038:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788853896 with bad export cookie 15496808721011131713 [10731.510177] LustreError: 321038:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10731.515422] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10731.875371] Lustre: server umount lustre-MDT0001 complete [10747.631920] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [10752.131658] Key type lgssc unregistered [10752.590579] LNet: 326589:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10752.601856] LNetError: 326589:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10752.624202] LNet: Removed LNI 192.168.202.111@tcp [10753.805687] Key type .llcrypt unregistered [10753.808198] Key type ._llcrypt unregistered [10780.085966] Key type ._llcrypt registered [10780.088334] Key type .llcrypt registered [10780.187657] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [10781.525274] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10781.650620] alg: No test for adler32 (adler32-zlib) [10782.698724] Lustre: Lustre: Build Version: 2.17.57_111_g33f3974 [10782.967276] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [10784.663284] Key type lgssc registered [10785.934343] Lustre: Echo OBD driver; http://www.lustre.org/ [10793.538735] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10793.584245] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10795.026929] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10799.260688] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10805.219311] LustreError: 328006:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10805.231355] LustreError: 328006:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10805.796194] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10810.350488] LustreError: 328005:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10812.949319] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10813.298082] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10818.632030] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10830.808322] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10831.436959] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10838.630213] LustreError: 329685:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10838.706320] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10840.677133] LustreError: 329686:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10840.699980] LustreError: 329686:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10840.703026] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10845.299433] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10846.187070] LustreError: 330041:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10846.202985] LustreError: 330041:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10852.294300] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10852.442399] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10857.463819] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10859.227780] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10869.014488] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10877.465809] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10877.973981] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10985.442645] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10985.448686] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10985.471678] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10990.569212] LustreError: 329686:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10990.591787] LustreError: 329686:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10990.644452] Lustre: server umount lustre-OST0001 complete [10997.729843] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10997.747767] LustreError: Skipped 1 previous similar message [10997.755877] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10997.770489] Lustre: Skipped 1 previous similar message [10997.783617] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10997.795410] Lustre: Skipped 1 previous similar message [10999.778702] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10999.790308] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10999.807601] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11002.199582] Lustre: server umount lustre-OST0000 complete [11005.922185] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11005.927345] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11005.939240] Lustre: Skipped 1 previous similar message [11005.959552] Lustre: Skipped 3 previous similar messages [11011.043686] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11011.055246] Lustre: Skipped 1 previous similar message [11020.256096] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11020.627630] Lustre: server umount lustre-MDT0000 complete [11021.288336] LustreError: 331157:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11021.317243] LustreError: 331157:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [11025.188090] LustreError: 327987:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788854190 with bad export cookie 2004861325972647136 [11025.201784] LustreError: 327987:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11025.206406] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11025.906310] Lustre: server umount lustre-MDT0001 complete [11039.331625] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [11044.361328] Key type lgssc unregistered [11045.016862] LNet: 332953:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11045.046400] LNetError: 332953:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11045.084293] LNet: Removed LNI 192.168.202.111@tcp [11046.703523] Key type .llcrypt unregistered [11046.705811] Key type ._llcrypt unregistered [11093.206957] Key type ._llcrypt registered [11093.208800] Key type .llcrypt registered [11093.296398] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [11094.390620] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11094.487076] alg: No test for adler32 (adler32-zlib) [11095.595139] Lustre: Lustre: Build Version: 2.17.57_111_g33f3974 [11095.853467] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [11097.655178] Key type lgssc registered [11098.685646] Lustre: Echo OBD driver; http://www.lustre.org/ [11107.913933] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11107.941962] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11109.282283] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11113.366635] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11119.587645] LustreError: 335238:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11119.608755] LustreError: 335238:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11123.282481] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11123.658839] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11129.231306] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11132.995793] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11141.479096] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11141.911123] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11148.078103] LustreError: 336728:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11148.282803] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11153.399534] LustreError: 336727:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11154.436897] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11157.071361] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11157.439317] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11162.626853] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11164.576628] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11176.195811] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11185.225257] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11203.556557] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11203.561195] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11203.577783] Lustre: Skipped 3 previous similar messages [11203.594543] Lustre: Skipped 3 previous similar messages [11207.409327] Lustre: server umount lustre-MDT0000 complete [11208.682761] LustreError: 335238:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11208.723983] LustreError: 335238:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [11211.728493] LustreError: 338247:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788854376 with bad export cookie 10433479502030974595 [11211.729802] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11211.735742] LustreError: 338247:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11212.366396] Lustre: server umount lustre-MDT0001 complete [11226.768309] Lustre: server umount lustre-OST0000 complete [11229.664749] Lustre: 334501:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788854378/real 1788854378] req@ffffa0fbf7334700 x1875749644603392/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788854394 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11229.708280] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11231.589746] Lustre: server umount lustre-OST0001 complete [11249.589974] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [11265.519647] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11266.038990] LustreError: 340504:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11266.095242] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11272.513458] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11276.261443] LustreError: 340505:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11276.299992] LustreError: 340505:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11283.065505] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11283.458646] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11290.023071] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11293.410787] Lustre: 341645:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11301.815953] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11302.248468] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11308.434169] LustreError: 342012:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11308.469786] LustreError: 342012:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11310.726085] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11311.549410] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11321.036395] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11326.474307] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11328.219174] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11336.358989] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11340.845415] Lustre: 343531:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11346.925878] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11346.934929] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11346.956106] Lustre: Skipped 2 previous similar messages [11346.985891] Lustre: Skipped 3 previous similar messages [11352.072067] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11352.078400] Lustre: Skipped 1 previous similar message [11352.463314] Lustre: server umount lustre-MDT0000 complete [11356.509754] LustreError: 340485:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788854521 with bad export cookie 10433479502030976296 [11356.527576] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11356.528395] LustreError: 340485:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11356.909462] Lustre: server umount lustre-MDT0001 complete [11370.155865] Lustre: server umount lustre-OST0000 complete [11373.535834] Lustre: 334503:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788854522/real 1788854522] req@ffffa0fbc2c78e00 x1875749644656000/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788854538 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11373.572648] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11373.599534] Lustre: Skipped 1 previous similar message [11375.646905] Lustre: server umount lustre-OST0001 complete [11380.588191] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11382.332431] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11384.714058] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 04:02:28 (1788854548) [11412.859366] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [11422.902507] loop0: detected capacity change from 0 to 1099511627776 [11423.937725] SGI XFS with ACLs, security attributes, quota, no debug enabled [11423.945907] *** VALIDATE xfs *** [11423.962908] XFS (loop0): Mounting V5 Filesystem [11423.996727] XFS (loop0): Ending clean mount [11425.573639] loop1: detected capacity change from 0 to 18691697672192 [11570.932972] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [11582.632987] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [11625.786986] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11625.963534] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11625.996588] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11626.066469] Lustre: lustre-MDT0000: new disk, initializing [11626.137065] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11626.143700] Lustre: Skipped 1 previous similar message [11626.156217] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11631.212167] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11644.893864] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11645.110352] Lustre: 348693:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11645.219067] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11645.237661] Lustre: Skipped 1 previous similar message [11645.400786] Lustre: lustre-MDT0001: new disk, initializing [11645.662792] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11645.678494] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11651.997517] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11663.213795] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11670.693385] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11679.199199] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11679.558791] Lustre: lustre-OST0000: new disk, initializing [11679.571412] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11679.594278] Lustre: 350662:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11680.713185] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11680.717703] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11680.856703] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11686.341398] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11697.183785] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11702.267814] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11702.468468] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11707.936630] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11708.182425] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11711.458059] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11711.466384] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11711.483318] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11711.490730] LustreError: Skipped 1 previous similar message [11711.509464] Lustre: Skipped 1 previous similar message [11716.475142] Lustre: server umount lustre-OST0000 complete [11721.698159] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11721.708752] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11721.725224] Lustre: Skipped 2 previous similar messages [11721.750417] Lustre: Skipped 1 previous similar message [11727.809561] Lustre: server umount lustre-MDT0000 complete [11730.913535] LustreError: 349534:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11730.942241] LustreError: 349534:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [11732.396797] LustreError: 348686:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788854897 with bad export cookie 10433479502030977850 [11732.405488] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11732.411732] LustreError: 348686:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11732.945775] Lustre: server umount lustre-MDT0001 complete [11741.282615] XFS (loop0): Unmounting Filesystem [11747.790716] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 04:08:31 (1788854911) [11756.417562] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11757.007570] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11757.016433] Lustre: Skipped 2 previous similar messages [11763.082176] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11774.157070] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11779.233950] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11789.032780] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11795.101961] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11802.588809] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11810.506993] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11818.387615] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11843.040303] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11843.052822] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11843.058336] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11843.058345] Lustre: Skipped 2 previous similar messages [11843.090367] Lustre: Skipped 1 previous similar message [11847.103264] Lustre: server umount lustre-OST0000 complete [11858.957220] Lustre: server umount lustre-MDT0000 complete [11863.521069] LustreError: 353219:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11863.540277] LustreError: 353219:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [11863.970519] LustreError: 353206:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788855029 with bad export cookie 10433479502030978725 [11863.975481] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11863.979347] LustreError: 353206:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11864.420143] Lustre: server umount lustre-MDT0001 complete [11877.641884] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [11881.896597] Key type lgssc unregistered [11882.319663] LNet: 356877:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11882.331610] LNetError: 356877:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11882.368517] LNet: Removed LNI 192.168.202.111@tcp [11883.390233] Key type .llcrypt unregistered [11883.392731] Key type ._llcrypt unregistered [11916.244594] Key type ._llcrypt registered [11916.249237] Key type .llcrypt registered [11916.527805] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [11919.226545] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11919.265421] alg: No test for adler32 (adler32-zlib) [11920.786301] Lustre: Lustre: Build Version: 2.17.57_111_g33f3974 [11921.567763] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [11923.383610] Key type lgssc registered [11925.614551] Lustre: Echo OBD driver; http://www.lustre.org/ [11935.137602] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11935.159469] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11936.794855] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11941.408352] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11945.960140] LustreError: 358443:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11945.983950] LustreError: 358443:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11950.513284] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11951.010326] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11957.327422] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11965.445320] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11971.789604] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11980.400394] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11980.682656] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11987.893761] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11996.069937] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12005.364761] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12005.377389] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12005.392632] Lustre: Skipped 1 previous similar message [12009.157421] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.11@tcp (stopping) [12010.466384] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12010.472706] Lustre: Skipped 1 previous similar message [12010.849547] Lustre: server umount lustre-MDT0001 complete [12014.255669] LustreError: 360338:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.11@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12015.585816] LustreError: 358438:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12015.621503] LustreError: 358438:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12019.363601] LustreError: 358439:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.11@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12022.265512] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12022.469892] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12022.475565] Lustre: Found index 1 for lustre-MDT0001, updating log [12022.529919] Lustre: 358433:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12022.972688] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12028.390723] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [12028.402993] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12032.008302] LustreError: 362044:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12032.018818] LustreError: 362044:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12032.057191] LustreError: 358435:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12032.156059] LustreError: 362047:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.111@tcp: cfg command failed: rc = -17 [12032.171401] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [12032.171401] [12335.592876] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12335.605599] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12335.631109] Lustre: Skipped 1 previous similar message [12335.656655] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12338.657696] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12340.895803] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.11@tcp (stopping) [12340.911699] Lustre: Skipped 3 previous similar messages [12341.531959] Lustre: server umount lustre-MDT0001 complete [12343.790333] LustreError: 358439:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12343.808329] LustreError: 358439:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12354.017716] LustreError: 358444:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12354.037234] LustreError: 358444:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [12354.836698] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12355.039208] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12355.043023] Lustre: Found index 1 for lustre-MDT0001, updating log [12355.125385] Lustre: 358432:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12355.476410] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12360.510729] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [12360.695900] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12362.769979] LustreError: 369818:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12362.792413] LustreError: 369818:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12362.807616] LustreError: 358435:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12362.810318] LustreError: 358435:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12362.934636] LustreError: 369828:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12362.958841] LustreError: 369828:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12362.974495] LustreError: 369828:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.111@tcp: cfg command failed: rc = -17 [12362.993159] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.111@tcp [12362.993159] [12667.880349] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12667.881107] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12667.890212] Lustre: Skipped 1 previous similar message [12667.893750] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12667.912172] Lustre: Skipped 1 previous similar message [12672.176899] Lustre: server umount lustre-MDT0001 complete [12672.997700] LustreError: 360338:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12673.015150] LustreError: 360338:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12683.941978] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12684.088105] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12684.101378] Lustre: Found index 1 for lustre-MDT0001, updating log [12684.158682] Lustre: 358433:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12684.498968] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12689.389546] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [12689.929053] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12691.450376] LustreError: 377721:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12691.457106] LustreError: 377721:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12691.477398] LustreError: 358435:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12691.482303] LustreError: 358435:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12691.567613] LustreError: 377726:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12691.597913] LustreError: 377726:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.111@tcp: cfg command failed: rc = -114 [12691.614111] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12691.614111] [13012.453832] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13012.459044] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13012.459087] Lustre: Skipped 1 previous similar message [13012.504972] Lustre: Skipped 2 previous similar messages [13019.140654] Lustre: server umount lustre-MDT0000 complete [13022.690556] LustreError: 358438:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13022.711821] LustreError: 358438:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [13023.431598] LustreError: 377378:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788856188 with bad export cookie 6908865410953490138 [13023.434033] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13023.448914] LustreError: 377378:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [13027.809739] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13030.105638] Lustre: server umount lustre-MDT0001 complete [13035.254842] Lustre: server umount lustre-OST0000 complete [13046.745807] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [13066.781094] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [13110.984572] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 04:31:15 (1788856275) [13118.308430] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13118.646819] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13118.691671] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13118.807444] Lustre: lustre-MDT0000: new disk, initializing [13118.901555] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13118.920285] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13124.310606] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [13136.253261] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13136.405416] Lustre: 387849:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13136.476744] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13136.480314] Lustre: Skipped 1 previous similar message [13136.558867] Lustre: lustre-MDT0001: new disk, initializing [13136.676439] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13136.727548] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13136.741800] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13141.647203] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [13152.101727] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13157.903563] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13165.831975] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13166.315794] Lustre: lustre-OST0000: new disk, initializing [13166.321553] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13166.347648] Lustre: 389819:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13166.486269] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13174.452688] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [13174.897099] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13174.906847] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13175.141835] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13185.474885] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13195.744806] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13195.759225] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13195.759386] Lustre: Skipped 1 previous similar message [13195.793351] Lustre: Skipped 9 previous similar messages [13200.232643] Lustre: server umount lustre-MDT0000 complete [13200.866548] LustreError: 387856:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13200.917503] LustreError: 387856:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [13204.891545] LustreError: 387841:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788856370 with bad export cookie 6908865410953492084 [13204.904716] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13204.907203] LustreError: 387841:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [13205.438469] Lustre: server umount lustre-MDT0001 complete [13221.303516] Lustre: server umount lustre-OST0000 complete [13258.190697] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [13270.591266] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [13308.919799] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12973 sec ========= 04:34:32 (1788856472) [13310.676239] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 04:34:34 (1788856474) === [13315.498870] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 04:34:39 (1788856479) === [13359.590994] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [13363.059429] Key type lgssc unregistered [13363.320965] LNet: 396186:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13363.330956] LNetError: 396186:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13363.352108] LNet: Removed LNI 192.168.202.111@tcp [13364.334116] Key type .llcrypt unregistered [13364.336693] Key type ._llcrypt unregistered