[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 488175291 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2400.000 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.003352] x2apic enabled [ 0.004008] Switched APIC routing to physical x2apic. [ 0.005012] kvm-guest: setup PV IPIs [ 0.008415] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 0.009023] Calibrating delay loop (skipped) preset value.. 4800.00 BogoMIPS (lpj=2400000) [ 0.010019] pid_max: default: 32768 minimum: 301 [ 0.011131] LSM: Security Framework initializing [ 0.012062] Yama: becoming mindful. [ 0.013044] SELinux: Initializing. [ 0.014080] *** VALIDATE selinux *** [ 0.022707] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027028] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028185] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029120] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031104] *** VALIDATE tmpfs *** [ 0.032443] *** VALIDATE proc *** [ 0.034135] *** VALIDATE cgroup *** [ 0.035010] *** VALIDATE cgroup2 *** [ 0.036267] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.037150] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.038008] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.039030] Spectre V2 : User space: Vulnerable [ 0.040009] Speculative Store Bypass: Vulnerable [ 0.043271] debug: unmapping init [mem 0xffffffffa0c59000-0xffffffffa0c60fff] [ 0.045887] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.046698] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.047021] ... version: 2 [ 0.048011] ... bit width: 48 [ 0.049009] ... generic registers: 4 [ 0.050012] ... value mask: 0000ffffffffffff [ 0.051012] ... max period: 00007fffffffffff [ 0.052013] ... fixed-purpose events: 3 [ 0.053014] ... event mask: 000000070000000f [ 0.054293] rcu: Hierarchical SRCU implementation. [ 0.056587] smp: Bringing up secondary CPUs ... [ 0.057607] x86: Booting SMP configuration: [ 0.058033] .... node #0, CPUs: #1 #2 #3 [ 0.063015] smp: Brought up 1 node, 4 CPUs [ 0.065018] smpboot: Max logical packages: 1 [ 0.066013] smpboot: Total of 4 processors activated (19200.00 BogoMIPS) [ 0.139707] node 0 deferred pages initialised in 72ms [ 0.143422] devtmpfs: initialized [ 0.145280] x86/mm: Memory block size: 128MB [ 0.147299] gcov: version magic: 0x41383552 [ 0.150239] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.153093] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.156275] pinctrl core: initialized pinctrl subsystem [ 0.157185] [ 0.157532] ************************************************************* [ 0.160010] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.162011] ** ** [ 0.163007] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.164009] ** ** [ 0.166013] ** This means that this kernel is built to expose internal ** [ 0.169011] ** IOMMU data structures, which may compromise security on ** [ 0.171010] ** your system. ** [ 0.174014] ** ** [ 0.177014] ** If you see this message and you are not debugging the ** [ 0.179012] ** kernel, report this immediately to your vendor! ** [ 0.182020] ** ** [ 0.184013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.187017] ************************************************************* [ 0.189692] NET: Registered protocol family 16 [ 0.191451] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.194071] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.196057] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.200045] cpuidle: using governor menu [ 0.202101] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.205732] PCI: Using configuration type 1 for base access [ 0.207186] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.214119] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.217070] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.221071] cryptd: max_cpu_qlen set to 1000 [ 0.223260] ACPI: Added _OSI(Module Device) [ 0.225047] ACPI: Added _OSI(Processor Device) [ 0.227038] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.229030] ACPI: Added _OSI(Processor Aggregator Device) [ 0.236311] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.244463] ACPI: Interpreter enabled [ 0.246075] ACPI: PM: (supports S0 S3 S4 S5) [ 0.248017] ACPI: Using IOAPIC for interrupt routing [ 0.249107] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.253421] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.266628] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.269073] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.273025] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.277090] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.282762] acpiphp: Slot [2] registered [ 0.284178] acpiphp: Slot [5] registered [ 0.286147] acpiphp: Slot [6] registered [ 0.288116] acpiphp: Slot [7] registered [ 0.289101] acpiphp: Slot [8] registered [ 0.291099] acpiphp: Slot [9] registered [ 0.292160] acpiphp: Slot [10] registered [ 0.294181] acpiphp: Slot [3] registered [ 0.296153] acpiphp: Slot [4] registered [ 0.297143] acpiphp: Slot [11] registered [ 0.299160] acpiphp: Slot [12] registered [ 0.301150] acpiphp: Slot [13] registered [ 0.302136] acpiphp: Slot [14] registered [ 0.304179] acpiphp: Slot [15] registered [ 0.305156] acpiphp: Slot [16] registered [ 0.307165] acpiphp: Slot [17] registered [ 0.309296] acpiphp: Slot [18] registered [ 0.311164] acpiphp: Slot [19] registered [ 0.313173] acpiphp: Slot [20] registered [ 0.314138] acpiphp: Slot [21] registered [ 0.316413] acpiphp: Slot [22] registered [ 0.317177] acpiphp: Slot [23] registered [ 0.319163] acpiphp: Slot [24] registered [ 0.321177] acpiphp: Slot [25] registered [ 0.322209] acpiphp: Slot [26] registered [ 0.323193] acpiphp: Slot [27] registered [ 0.325126] acpiphp: Slot [28] registered [ 0.327159] acpiphp: Slot [29] registered [ 0.328095] acpiphp: Slot [30] registered [ 0.330124] acpiphp: Slot [31] registered [ 0.331074] PCI host bridge to bus 0000:00 [ 0.332021] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.333022] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.336033] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.338028] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.340026] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.343026] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.344215] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.346877] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.348984] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.360934] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.365014] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.367021] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.369015] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.371016] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.373864] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.377000] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.380066] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.385341] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.391022] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.406024] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.412962] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.418647] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.425018] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.433016] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.449017] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.458571] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.472022] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.484020] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.507019] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.515644] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.523015] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.529014] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.545018] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.558983] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.567019] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.574017] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.588018] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.598749] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.607013] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.612017] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.625024] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.639967] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.645015] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.654016] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.675020] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.689088] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.692473] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.694392] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.697438] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.700238] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.705040] iommu: Default domain type: Passthrough [ 0.707309] SCSI subsystem initialized [ 0.709152] ACPI: bus type USB registered [ 0.710128] usbcore: registered new interface driver usbfs [ 0.713106] usbcore: registered new interface driver hub [ 0.715120] usbcore: registered new device driver usb [ 0.717177] pps_core: LinuxPPS API ver. 1 registered [ 0.719011] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.722082] PTP clock support registered [ 0.725076] EDAC MC: Ver: 3.0.0 [ 0.727132] PCI: Using ACPI for IRQ routing [ 0.728889] NetLabel: Initializing [ 0.730015] NetLabel: domain hash size = 128 [ 0.732012] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.734094] NetLabel: unlabeled traffic allowed by default [ 0.737131] vgaarb: loaded [ 0.739297] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.741016] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.753682] clocksource: Switched to clocksource kvm-clock [ 0.862807] VFS: Disk quotas dquot_6.6.0 [ 0.864569] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.867464] *** VALIDATE ramfs *** [ 0.869062] *** VALIDATE hugetlbfs *** [ 0.871803] pnp: PnP ACPI init [ 0.874383] pnp: PnP ACPI: found 6 devices [ 0.906641] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.909811] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.912464] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.915777] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.918910] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.922027] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.926026] NET: Registered protocol family 2 [ 0.929131] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.935096] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.939959] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.946307] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.949896] TCP: Hash tables configured (established 65536 bind 65536) [ 0.953176] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.956750] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.960020] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.963280] NET: Registered protocol family 1 [ 0.966871] RPC: Registered named UNIX socket transport module. [ 0.969529] RPC: Registered udp transport module. [ 0.971455] RPC: Registered tcp transport module. [ 0.973216] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.975741] NET: Registered protocol family 44 [ 0.977352] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.979330] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.981352] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.984100] PCI: CLS 0 bytes, default 64 [ 0.986159] Unpacking initramfs... [ 2.379328] debug: unmapping init [mem 0xffff9677bcc54000-0xffff9677bffbffff] [ 2.384595] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.388167] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.391141] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 2.877451] Initialise system trusted keyrings [ 2.878941] Key type blacklist registered [ 2.880477] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.911199] zbud: loaded [ 2.918549] *** VALIDATE nfs *** [ 2.921197] *** VALIDATE nfs4 *** [ 2.923437] pstore: using deflate compression [ 2.928188] Platform Keyring initialized [ 3.030861] NET: Registered protocol family 38 [ 3.032795] Key type asymmetric registered [ 3.034212] Asymmetric key parser 'x509' registered [ 3.035962] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.039157] io scheduler mq-deadline registered [ 3.040591] io scheduler kyber registered [ 3.041941] io scheduler bfq registered [ 3.043961] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.047262] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.049303] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.052529] ACPI: Power Button [PWRF] [ 3.057590] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.065299] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.080610] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.088308] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.117201] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.149402] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.180918] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.185831] Non-volatile memory driver v1.3 [ 3.189480] Linux agpgart interface v0.103 [ 3.222848] virtio_blk virtio1: [vda] 146008 512-byte logical blocks (74.8 MB/71.3 MiB) [ 3.225621] vda: detected capacity change from 0 to 74756096 [ 3.239716] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.242876] vdb: detected capacity change from 0 to 1073741824 [ 3.263149] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.266736] vdc: detected capacity change from 0 to 2621440000 [ 3.287052] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.290412] vdd: detected capacity change from 0 to 2621440000 [ 3.311775] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.315159] vde: detected capacity change from 0 to 4294967296 [ 3.336846] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.339359] vdf: detected capacity change from 0 to 4294967296 [ 3.347501] libphy: Fixed MDIO Bus: probed [ 3.358341] usbcore: registered new interface driver usbserial_generic [ 3.360885] usbserial: USB Serial support registered for generic [ 3.362912] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.366929] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.368727] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.371092] mousedev: PS/2 mouse device common for all mice [ 3.374040] rtc_cmos 00:05: RTC can wake from S4 [ 3.376404] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.376702] rtc_cmos 00:05: registered as rtc0 [ 3.381688] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.381986] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.386918] intel_pstate: CPU model not supported [ 3.389329] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.393252] hid: raw HID events driver (C) Jiri Kosina [ 3.395282] usbcore: registered new interface driver usbhid [ 3.397137] usbhid: USB HID core driver [ 3.398586] drop_monitor: Initializing network drop monitor service [ 3.400750] Initializing XFRM netlink socket [ 3.402652] NET: Registered protocol family 10 [ 3.405912] Segment Routing with IPv6 [ 3.407845] NET: Registered protocol family 17 [ 3.410479] mpls_gso: MPLS GSO support [ 3.416152] RAS: Correctable Errors collector initialized. [ 3.418170] AVX version of gcm_enc/dec engaged. [ 3.419667] AES CTR mode by8 optimization enabled [ 3.504081] sched_clock: Marking stable (3504043111, 0)->(4416299613, -912256502) [ 3.507734] registered taskstats version 1 [ 3.509855] Loading compiled-in X.509 certificates [ 3.512264] zswap: loaded using pool lzo/zbud [ 3.542451] Key type big_key registered [ 3.556177] Key type encrypted registered [ 3.557879] ima: No TPM chip found, activating TPM-bypass! [ 3.560028] ima: Allocated hash algorithm: sha1 [ 3.561943] ima: No architecture policies found [ 3.563913] evm: Initialising EVM extended attributes: [ 3.565880] evm: security.selinux [ 3.567194] evm: security.ima [ 3.568317] evm: security.capability [ 3.569496] evm: HMAC attrs: 0x1 [ 3.572048] rtc_cmos 00:05: setting system clock to 2026-08-22 01:01:34 UTC (1787360494) [ 3.577752] debug: unmapping init [mem 0xffffffffa1c03000-0xffffffffa1dfffff] [ 3.580788] debug: unmapping init [mem 0xffffffffa0982000-0xffffffffa0c58fff] [ 3.590099] Write protecting the kernel read-only data: 28672k [ 3.593381] debug: unmapping init [mem 0xffffffff9f003000-0xffffffff9f1fffff] [ 3.596308] debug: unmapping init [mem 0xffffffff9f914000-0xffffffff9f9fffff] [ 3.629363] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.640288] systemd[1]: Detected virtualization kvm. [ 3.643097] systemd[1]: Detected architecture x86-64. [ 3.645251] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.675873] systemd[1]: No hostname configured. [ 3.677780] systemd[1]: Set hostname to . [ 3.680378] random: systemd: uninitialized urandom read (16 bytes read) [ 3.683320] systemd[1]: Initializing machine ID from random generator. [ 3.717110] random: ln: uninitialized urandom read (6 bytes read) [ 3.830898] random: systemd: uninitialized urandom read (16 bytes read) [ 3.834338] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ 3.842387] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 3.848734] systemd[1]: Reached target Paths. [ OK ] Reached target Paths. [ OK ] Listening on Journal Socket. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Slices. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Local Encrypted Volumes. Starting Create Volatile Files and Directories... Starting Apply Kernel Variables... Starting Setup Virtual Console... [ OK ] Started Memstrack Anylazing Service. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on udev Control Socket. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Sockets. Starting Journal Service... [ OK ] Reached target Timers. [ OK ] Reached target Swap. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.549472] device-mapper: uevent: version 1.0.3 [ 4.552178] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.364615] virtio_net virtio0 ens2: renamed from eth0 [ 5.366586] random: fast init done [ 5.444498] scsi host0: ata_piix [ 5.491384] scsi host1: ata_piix [ 5.493340] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.496198] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.904832] dracut-initqueue[588]: RTNETLINK answers: File exists [ 10.075314] random: crng init done [ 10.076907] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.682105] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Timers. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Initrd Root Device. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. Stopping udev Kernel Device Manager... [ OK ] Stopped Apply Kernel Variables. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 12.098436] printk: systemd: 26 output lines suppressed due to ratelimiting [ 12.373525] SELinux: Disabled at runtime. [ 12.439201] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.457805] systemd[1]: Detected virtualization kvm. [ 12.459198] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.980381] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.985102] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.989768] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.996462] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 13.000696] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 13.009946] systemd[1]: Starting Journal Service... Starting Journal Service... [ 13.026925] systemd[1]: Listening on Process Core Dump Socket. [ OK ] Listening on Process Core Dump Socket. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Created slice User and Session Slice. [ OK ] Started Forward Password Requests to Wall Directory Watch. Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Reached target rpc_pipefs.target. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. Starting Remount Root and Kernel File Systems... Mounting Kernel Debug File System... [ OK ] Listening on udev Kernel Socket. [ 13.083829] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Mounting Huge Pages File System... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Created slice system-getty.slice. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Stopped target Initrd Root File System. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. Starting Apply Kernel Variables... Mounting POSIX Message Queue File System... [ OK ] Reached target Slices. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Started Journal Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Activated swap /dev/disk/by-label/SWAP. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted Huge Pages File System. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted POSIX Message Queue File System. Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Mounted /mnt. [ OK ] Started udev Coldplug all Devices. [ 13.467098] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.796762] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.828568] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 14.049064] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 14.086873] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (5s / no limit) [** ] A start job is running for Configur…-only root support (6s / no limit)[ 19.590538] Key type dns_resolver registered [*** ] A start job is running for Configur…-only root support (6s / no limit)[ 20.216623] NFS: Registering the id_resolver key type [ 20.219290] Key type id_resolver registered [ 20.222116] Key type id_legacy registered [ *** ] A start job is running for Configur…-only root support (7s / no limit) [ *** ] A start job is running for Configur…-only root support (7s / no limit) [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Load/Save Random Seed... Starting Create Volatile Files and Directories... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started dnf makecache --timer. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Reached target Basic System. Starting Login Service... Starting Restore /run/initramfs on shutdown... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Reached target sshd-keygen.target. [ OK ] Started irqbalance daemon. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg429-server login: [ 46.511836] hrtimer: interrupt took 10863266 ns [ 78.430665] libcfs: loading out-of-tree module taints kernel. [ 78.506503] Key type ._llcrypt registered [ 78.508504] Key type .llcrypt registered [ 78.606694] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [ 97.935380] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 99.514922] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 99.527796] alg: No test for adler32 (adler32-zlib) [ 101.028982] Lustre: Lustre: Build Version: 2.17.57_45_g4630995 [ 101.915274] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 103.688296] Key type lgssc registered [ 105.560538] Lustre: Echo OBD driver; http://www.lustre.org/ [ 124.642713] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 163.801871] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 178.230723] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 178.272401] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 179.611433] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 179.636095] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 179.709909] Lustre: lustre-MDT0000: new disk, initializing [ 179.781809] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 179.797621] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 184.636100] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 197.530862] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 197.679318] Lustre: 6509:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 197.737256] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 197.745306] Lustre: Skipped 1 previous similar message [ 197.865463] Lustre: lustre-MDT0001: new disk, initializing [ 197.988503] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 198.038779] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 198.052671] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 202.609373] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 208.089729] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 218.364859] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 218.685491] Lustre: lustre-OST0000: new disk, initializing [ 218.693452] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 218.701762] Lustre: 8448:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 218.765110] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 220.543024] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 220.553197] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 220.634686] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 225.287977] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 239.530569] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 239.663284] Lustre: lustre-OST0001: new disk, initializing [ 239.666402] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 239.674614] Lustre: 9522:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 239.739872] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 246.030611] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 247.878731] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 247.895808] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 247.960164] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 258.927228] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 266.689912] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 272.865986] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing check_logdir /tmp/testlogs/ [ 277.299261] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing yml_node [ 281.893555] Lustre: DEBUG MARKER: Client: 2.17.57.45 [ 284.470546] Lustre: DEBUG MARKER: MDS: 2.17.57.45 [ 287.204559] Lustre: DEBUG MARKER: OSS: 2.17.57.45 [ 288.977554] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Fri Aug 21 21:06:18 EDT 2026 [ 305.385740] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 306.910574] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 325.089187] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 325.092033] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 325.103254] Lustre: Skipped 2 previous similar messages [ 325.117880] Lustre: Skipped 3 previous similar messages [ 329.085954] Lustre: server umount lustre-MDT0000 complete [ 332.538423] LustreError: 6503:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787360823 with bad export cookie 8811866899425955983 [ 332.540090] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 332.547459] LustreError: 6503:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 335.338520] LustreError: 8830:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 335.338916] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 335.339940] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 335.353580] LustreError: 8830:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 335.359762] Lustre: Skipped 2 previous similar messages [ 337.377069] LustreError: 6518:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 337.378146] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 337.407565] LustreError: 6518:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 337.415472] Lustre: Skipped 2 previous similar messages [ 338.994542] Lustre: server umount lustre-MDT0001 complete [ 343.743424] Lustre: server umount lustre-OST0000 complete [ 348.153719] Lustre: server umount lustre-OST0001 complete [ 356.318750] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [ 366.048729] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 401.769841] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 402.075608] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 402.115460] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 402.236794] Lustre: lustre-MDT0000: new disk, initializing [ 402.336135] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 402.352241] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 407.282523] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 418.366979] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 418.445972] Lustre: 15078:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 418.487962] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 418.491109] Lustre: Skipped 1 previous similar message [ 418.557136] Lustre: lustre-MDT0001: new disk, initializing [ 418.654676] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 418.681253] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 418.688427] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 423.624558] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 434.468561] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 440.937292] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 448.172888] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 448.451354] Lustre: lustre-OST0000: new disk, initializing [ 448.457623] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 448.464218] Lustre: 17047:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 448.589805] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 449.706407] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 449.724544] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 449.890782] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 455.629216] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 464.767351] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 468.874720] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 469.110987] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 473.383423] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 473.560615] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 475.617947] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 475.630740] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 475.647443] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 475.652445] Lustre: Skipped 1 previous similar message [ 480.737803] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 480.744212] Lustre: Skipped 1 previous similar message [ 481.476201] Lustre: server umount lustre-OST0000 complete [ 485.858503] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 485.887051] Lustre: Skipped 1 previous similar message [ 490.465469] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 490.476269] LustreError: Skipped 1 previous similar message [ 490.489361] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 490.504155] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 490.508267] Lustre: Skipped 1 previous similar message [ 491.804963] Lustre: server umount lustre-MDT0000 complete [ 495.358042] LustreError: 15070:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787360986 with bad export cookie 8811866899425957516 [ 495.363140] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 495.378816] LustreError: 15070:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 495.620350] Lustre: server umount lustre-MDT0001 complete [ 500.306473] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 21:09:49 (1787360989) [ 506.618372] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 507.008754] LustreError: 19312:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 507.097705] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 511.653814] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 512.482444] LustreError: 19313:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 516.588042] LustreError: 19312:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 520.090046] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 525.095202] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 531.627455] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 536.842693] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 542.817966] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 548.986385] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 556.434960] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 559.894857] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 566.756828] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 566.765036] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 566.769136] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 566.782480] Lustre: Skipped 2 previous similar messages [ 566.802358] Lustre: Skipped 3 previous similar messages [ 570.549467] Lustre: server umount lustre-MDT0000 complete [ 571.873384] LustreError: 20040:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 571.890500] LustreError: 20040:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 573.991814] LustreError: 19293:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787361064 with bad export cookie 8811866899425958398 [ 573.996951] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 574.001157] LustreError: 19293:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 574.329861] Lustre: server umount lustre-MDT0001 complete [ 587.367683] Lustre: server umount lustre-OST0000 complete [ 609.655876] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 621.165117] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 621.628939] LustreError: 23693:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 621.741745] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 621.764529] Lustre: Skipped 2 previous similar messages [ 627.319178] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 636.411930] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 642.158125] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 645.552470] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 652.214520] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 658.280578] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 666.388979] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 666.549537] Lustre: lustre-OST0001: new disk, initializing [ 666.553345] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 666.560488] Lustre: 26009:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 668.631248] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 668.637958] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 668.754626] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 672.055871] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 680.519962] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 684.397607] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 698.339350] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 698.359850] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 698.384500] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 702.671817] Lustre: server umount lustre-MDT0000 complete [ 704.490180] LustreError: 23693:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 704.513117] LustreError: 23693:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 706.328769] LustreError: 26008:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787361197 with bad export cookie 8811866899425959238 [ 706.334376] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 706.342147] LustreError: 26008:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 706.618379] Lustre: server umount lustre-MDT0001 complete [ 721.012370] Lustre: server umount lustre-OST0000 complete [ 734.493544] Lustre: server umount lustre-OST0001 complete [ 751.083856] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 762.633054] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 763.560546] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 763.576651] Lustre: Skipped 3 previous similar messages [ 768.993964] LustreError: 29030:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 769.023187] LustreError: 29030:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 770.974619] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 782.887000] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 788.959171] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 792.083925] Lustre: 30180:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 799.922084] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 806.765402] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 816.229086] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 824.511731] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 833.150836] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 837.276826] Lustre: 32059:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 844.774298] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 844.780343] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 844.789362] Lustre: Skipped 3 previous similar messages [ 844.795152] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 844.798962] Lustre: Skipped 4 previous similar messages [ 850.831996] Lustre: server umount lustre-MDT0000 complete [ 855.301359] LustreError: 29010:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787361346 with bad export cookie 8811866899425960540 [ 855.306168] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 855.315741] LustreError: 29010:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 855.609910] Lustre: server umount lustre-MDT0001 complete [ 869.887762] Lustre: server umount lustre-OST0000 complete [ 873.953157] Lustre: 3646:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787361348/real 1787361348] req@ffff96783d037480 x1874183420406144/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787361364 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 875.353954] Lustre: server umount lustre-OST0001 complete [ 888.367705] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 21:16:16 (1787361376) [ 920.858892] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 930.817695] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 931.256169] LustreError: 34863:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 931.275751] LustreError: 34863:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 10 previous similar messages [ 936.225532] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 945.976463] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 952.277917] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 955.838072] Lustre: 36013:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 965.016644] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 972.901844] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 983.103129] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 990.018631] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 997.652062] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1000.847924] Lustre: 37894:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1007.588104] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1007.596597] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1007.603272] Lustre: Skipped 4 previous similar messages [ 1007.606426] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1007.610709] Lustre: Skipped 4 previous similar messages [ 1011.989442] Lustre: server umount lustre-MDT0000 complete [ 1015.774364] LustreError: 34845:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787361506 with bad export cookie 8811866899425961646 [ 1015.778982] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1015.788641] LustreError: 34845:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1016.121571] Lustre: server umount lustre-MDT0001 complete [ 1030.199907] Lustre: server umount lustre-OST0000 complete [ 1043.942459] Lustre: server umount lustre-OST0001 complete [ 1053.274567] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 21:19:01 (1787361541) [ 1072.590758] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 1083.015995] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1083.523396] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1083.534576] Lustre: Skipped 7 previous similar messages [ 1088.105793] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1097.792220] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1102.564857] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1105.223863] Lustre: 41188:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1112.735519] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1120.796941] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1130.139218] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1138.051565] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1147.770460] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1152.003229] Lustre: 43065:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1153.129605] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1153.134082] Lustre: Skipped 1 previous similar message [ 1156.075985] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1156.098538] Lustre: Skipped 6 previous similar messages [ 1162.337072] Lustre: server umount lustre-MDT0000 complete [ 1166.833731] LustreError: 43067:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787361657 with bad export cookie 8811866899425962752 [ 1166.850681] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1173.366972] Lustre: server umount lustre-MDT0001 complete [ 1181.750921] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1192.375940] LustreError: 41551:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1192.401526] LustreError: 41551:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 25 previous similar messages [ 1197.458572] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1202.671060] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1206.083684] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1211.111895] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1211.571404] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1211.581511] Lustre: Skipped 1 previous similar message [ 1219.434491] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1221.021328] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1227.383176] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1229.276764] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1236.977337] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1240.698690] Lustre: server umount lustre-MDT0000 complete [ 1244.280198] LustreError: 40018:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787361735 with bad export cookie 8811866899425963879 [ 1244.287746] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1244.294446] LustreError: 40018:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 1244.644719] Lustre: server umount lustre-MDT0001 complete [ 1259.148961] Lustre: server umount lustre-OST0000 complete [ 1273.025495] Lustre: server umount lustre-OST0001 complete [ 1281.376825] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 21:22:50 (1787361770) [ 1297.078865] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 1307.288881] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1312.375340] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1321.586354] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1326.615295] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1329.668297] Lustre: 48825:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1337.325802] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1344.376666] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1354.840062] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1361.948461] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1370.945752] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1375.521707] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1388.843278] Lustre: server umount lustre-MDT0000 complete [ 1392.730393] LustreError: 47655:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787361883 with bad export cookie 8811866899425964670 [ 1392.732714] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1392.746795] LustreError: 47655:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1393.161822] Lustre: server umount lustre-MDT0001 complete [ 1406.930793] Lustre: server umount lustre-OST0000 complete [ 1420.951949] Lustre: server umount lustre-OST0001 complete [ 1431.496772] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 21:25:20 (1787361920) [ 1434.445209] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1436.051310] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 21:25:25 (1787361925) [ 1448.427938] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [ 1457.567447] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 1478.626047] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1478.902252] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1478.996642] Lustre: lustre-MDT0000: new disk, initializing [ 1479.066275] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1483.684960] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1495.739271] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1505.171499] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1505.569874] Lustre: lustre-OST0000: new disk, initializing [ 1505.574355] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1505.581395] Lustre: Skipped 1 previous similar message [ 1505.589868] Lustre: 54902:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1507.783805] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1507.795584] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1507.843460] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1513.709761] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1523.822831] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1526.802308] Lustre: DEBUG MARKER: create test files [ 1564.132363] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1564.144935] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1564.164788] Lustre: Skipped 10 previous similar messages [ 1564.170496] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1564.208368] Lustre: Skipped 23 previous similar messages [ 1568.540436] Lustre: server umount lustre-OST0000 complete [ 1574.460133] Lustre: server umount lustre-MDT0000 complete [ 1587.380838] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 1591.209239] Key type lgssc unregistered [ 1591.613163] LNet: 56535:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1591.620380] LNetError: 56535:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1591.651255] LNet: Removed LNI 192.168.204.129@tcp [ 1592.807867] Key type .llcrypt unregistered [ 1592.810794] Key type ._llcrypt unregistered [ 1626.310598] Key type ._llcrypt registered [ 1626.316603] Key type .llcrypt registered [ 1626.442613] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 1627.646858] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1627.696469] alg: No test for adler32 (adler32-zlib) [ 1628.738642] Lustre: Lustre: Build Version: 2.17.57_45_g4630995 [ 1629.017553] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 1630.776367] Key type lgssc registered [ 1632.233088] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1641.797981] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1641.833738] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1643.388865] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1648.800848] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1656.769560] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1663.987167] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1664.405489] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1668.527249] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:101 to 0x240000400:129) [ 1668.660561] Lustre: 59536:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1668.673797] Lustre: 59536:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1668.681161] Lustre: 59536:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1668.686442] Lustre: 59536:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1668.692084] Lustre: 59536:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1668.697400] Lustre: 59536:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1670.451268] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1678.919799] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1683.977817] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1689.588409] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1698.698842] Lustre: server umount lustre-OST0000 complete [ 1712.356240] Lustre: server umount lustre-MDT0000 complete [ 1723.334232] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 1726.184772] Key type lgssc unregistered [ 1726.540638] LNet: 60887:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1726.545157] LNetError: 60887:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1726.565067] LNet: Removed LNI 192.168.204.129@tcp [ 1727.551990] Key type .llcrypt unregistered [ 1727.554200] Key type ._llcrypt unregistered [ 1762.584361] Key type ._llcrypt registered [ 1762.587258] Key type .llcrypt registered [ 1762.694838] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 1763.383194] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1763.405942] alg: No test for adler32 (adler32-zlib) [ 1764.539992] Lustre: Lustre: Build Version: 2.17.57_45_g4630995 [ 1764.753142] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 1766.400358] Key type lgssc registered [ 1767.601477] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1775.566149] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1775.599575] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1777.068829] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1781.631963] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1789.929187] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1796.790099] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1797.140693] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1801.289975] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1801.375618] Lustre: 63723:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1801.387736] Lustre: 63723:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1801.399143] Lustre: 63723:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1801.411993] Lustre: 63723:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1801.437318] Lustre: 63723:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1801.448590] Lustre: 63723:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1803.734847] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1812.083627] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1818.351820] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1831.911445] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1831.913427] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1831.920834] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1833.902261] Lustre: server umount lustre-OST0000 complete [ 1837.967374] Lustre: server umount lustre-MDT0000 complete [ 1848.163730] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 1851.007474] Key type lgssc unregistered [ 1851.346350] LNet: 65285:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1851.362715] LNetError: 65285:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1851.395878] LNet: Removed LNI 192.168.204.129@tcp [ 1852.381214] Key type .llcrypt unregistered [ 1852.384448] Key type ._llcrypt unregistered [ 1872.226956] Key type ._llcrypt registered [ 1872.233541] Key type .llcrypt registered [ 1872.351349] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [ 1887.840871] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 1888.907606] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1888.921884] alg: No test for adler32 (adler32-zlib) [ 1890.042557] Lustre: Lustre: Build Version: 2.17.57_45_g4630995 [ 1890.332483] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 1892.072176] Key type lgssc registered [ 1893.314780] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1933.359552] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1933.379870] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1934.605247] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1934.630561] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1934.707986] Lustre: lustre-MDT0000: new disk, initializing [ 1934.880624] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1934.917652] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1939.802299] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1951.320064] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1951.467584] Lustre: 68441:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1951.543772] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1951.549309] Lustre: Skipped 1 previous similar message [ 1951.630740] Lustre: lustre-MDT0001: new disk, initializing [ 1951.755057] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1951.799086] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1951.808695] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1956.316651] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1965.826612] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1972.630474] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1980.498855] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1980.831321] Lustre: lustre-OST0000: new disk, initializing [ 1980.840871] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1980.854577] Lustre: 70411:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1980.975592] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1987.681114] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1988.671558] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1988.683172] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1988.814398] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1998.018647] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2003.156990] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2003.441566] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2008.042788] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2008.279943] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2014.694455] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2014.704792] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2014.715966] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2014.723524] LustreError: Skipped 1 previous similar message [ 2016.800501] Lustre: server umount lustre-OST0000 complete [ 2023.394633] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2023.400832] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2023.413177] Lustre: Skipped 1 previous similar message [ 2023.419210] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2023.424206] Lustre: Skipped 1 previous similar message [ 2024.929387] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2024.932575] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2026.935439] Lustre: server umount lustre-MDT0000 complete [ 2031.060609] LustreError: 69235:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787362521 with bad export cookie 16990179954143615879 [ 2031.066183] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2031.066752] LustreError: 69235:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2031.379954] Lustre: server umount lustre-MDT0001 complete [ 2040.936645] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 21:35:29 (1787362529) [ 2052.723827] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 2095.144229] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [ 2106.643733] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 2147.200411] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2147.499830] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2147.526589] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2147.681492] Lustre: lustre-MDT0000: new disk, initializing [ 2147.788106] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2147.827240] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2152.841917] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 2163.556565] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2163.712456] Lustre: 76431:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2163.767511] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2163.772465] Lustre: Skipped 1 previous similar message [ 2163.849953] Lustre: lustre-MDT0001: new disk, initializing [ 2163.933606] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2163.963941] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2163.982503] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2168.306517] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 2177.811976] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2183.427398] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2191.327303] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2191.703378] Lustre: lustre-OST0000: new disk, initializing [ 2191.708088] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2191.715049] Lustre: 78397:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2191.858458] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2198.069403] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2198.091552] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2198.213710] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2198.632489] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 2209.340933] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2214.165636] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2214.403899] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2219.580497] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2219.779680] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2224.114061] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2224.114567] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2224.136026] LustreError: Skipped 1 previous similar message [ 2224.137117] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2224.137123] Lustre: Skipped 1 previous similar message [ 2224.190597] Lustre: Skipped 1 previous similar message [ 2229.219794] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2229.227701] Lustre: Skipped 2 previous similar messages [ 2235.872210] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2236.113375] Lustre: server umount lustre-OST0000 complete [ 2240.996632] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2241.013520] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2241.042361] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2241.050912] Lustre: Skipped 2 previous similar messages [ 2246.529971] Lustre: server umount lustre-MDT0000 complete [ 2249.704971] LustreError: 76442:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2249.716759] LustreError: 76442:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 2250.152243] LustreError: 78396:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787362741 with bad export cookie 16990179954143616761 [ 2250.159120] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2250.164391] LustreError: 78396:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2250.491157] Lustre: server umount lustre-MDT0001 complete [ 2258.230275] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 21:39:07 (1787362747) [ 2264.810412] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2265.411258] LustreError: 80810:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2265.501288] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2269.732434] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 2270.694814] LustreError: 80811:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2275.826446] LustreError: 80810:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2278.199289] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2283.553531] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 2291.091623] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2296.125330] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2302.493499] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2302.804573] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2302.813345] Lustre: Skipped 1 previous similar message [ 2308.675921] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 2316.597286] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2320.215765] Lustre: 83482:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting d738ba12-40bc-48b9-990c-f1fc479962ce at adminstrative request [ 2321.399962] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2321.412684] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2331.624026] Lustre: MGS: Client d738ba12-40bc-48b9-990c-f1fc479962ce (at 0@lo) reconnecting [ 2331.631058] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xebc93ea684b1626b to 0xebc93ea684b1650b [ 2331.647550] Lustre: MGC192.168.204.129@tcp: Connection restored to 0@lo (at 0@lo) [ 2358.239978] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2358.400762] Lustre: lustre-OST0001: new disk, initializing [ 2358.405761] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2358.411980] Lustre: 83875:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2365.682639] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 2367.545817] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2367.550827] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2367.615187] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2375.445378] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2382.824273] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2382.824308] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2382.836841] LustreError: Skipped 1 previous similar message [ 2382.838171] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2382.846741] Lustre: Skipped 2 previous similar messages [ 2382.859808] Lustre: Skipped 3 previous similar messages [ 2384.582951] Lustre: server umount lustre-OST0001 complete [ 2387.942916] LustreError: 82633:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2387.968656] LustreError: 82633:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2391.010487] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2398.196112] LustreError: 82632:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2398.212633] LustreError: 82632:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2403.307311] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2403.444496] Lustre: server umount lustre-OST0000 complete [ 2408.420145] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2408.449121] Lustre: Skipped 3 previous similar messages [ 2413.805713] Lustre: server umount lustre-MDT0000 complete [ 2418.025202] LustreError: 80791:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787362908 with bad export cookie 16990179954143618315 [ 2418.035987] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2418.044747] LustreError: 80791:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2418.458947] Lustre: server umount lustre-MDT0001 complete [ 2427.550500] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 21:41:56 (1787362916) [ 2429.651713] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2431.957922] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 21:42:00 (1787362920) [ 2433.816544] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2435.696685] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 21:42:04 (1787362924) [ 2437.341185] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2439.625831] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 21:42:08 (1787362928) [ 2441.218385] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2443.391972] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 21:42:12 (1787362932) [ 2445.377841] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2452.560490] loop: module loaded [ 2452.592545] loop0: detected capacity change from 0 to 4294967296 [ 2452.651868] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2454.445690] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2458.918830] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2463.507045] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2489.590592] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2489.631363] LustreError: 88002:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2489.636671] LustreError: 88002:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2489.641779] LustreError: 88002:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2489.646910] LustreError: 88002:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2508.146262] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [ 2518.848801] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 2557.003550] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2557.262314] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2557.285574] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2557.355456] Lustre: lustre-MDT0000: new disk, initializing [ 2557.413054] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2557.415818] Lustre: Skipped 1 previous similar message [ 2557.427228] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2562.039280] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 2572.342058] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2572.411439] Lustre: 90871:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2572.576037] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2577.279868] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 2588.294565] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2594.716373] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2601.949537] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2602.248431] Lustre: 92840:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2608.249157] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2608.407134] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2608.639960] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 2618.111842] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2623.194529] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2623.389439] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2628.122113] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2628.376915] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2634.210568] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2634.218503] LustreError: Skipped 1 previous similar message [ 2634.222818] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2634.230443] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2634.238503] Lustre: Skipped 1 previous similar message [ 2634.249831] Lustre: Skipped 12 previous similar messages [ 2636.420338] Lustre: server umount lustre-OST0000 complete [ 2646.543806] Lustre: server umount lustre-MDT0000 complete [ 2649.569188] LustreError: 93377:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2649.586725] LustreError: 93377:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2650.224817] LustreError: 90862:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787363141 with bad export cookie 16990179954143618931 [ 2650.231961] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2650.233576] LustreError: 90862:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2650.521371] Lustre: server umount lustre-MDT0001 complete [ 2660.429216] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 21:45:49 (1787363149) [ 2669.243659] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2676.307780] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 2685.921613] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2685.934692] LustreError: 95254:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2685.962665] LustreError: 95254:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2686.340411] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2686.348290] Lustre: Skipped 3 previous similar messages [ 2692.115460] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 2700.187115] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2706.009540] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2714.018782] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2719.985734] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 2728.368566] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2734.954227] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2735.239506] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2740.638508] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2740.874418] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2749.042110] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2749.243479] Lustre: lustre-OST0001: new disk, initializing [ 2749.245957] Lustre: Skipped 2 previous similar messages [ 2749.253611] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2749.260511] Lustre: Skipped 3 previous similar messages [ 2749.270295] Lustre: 98627:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2756.736796] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 2758.772747] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2758.782422] Lustre: Skipped 2 previous similar messages [ 2758.790304] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2758.899857] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2768.795988] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2774.328243] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2774.567148] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2779.869243] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2780.141333] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2800.473751] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2830.181603] Lustre: Failing over lustre-MDT0000 [ 2830.661601] Lustre: server umount lustre-MDT0000 complete [ 2830.827260] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2830.845084] Lustre: Skipped 3 previous similar messages [ 2830.853118] LustreError: 95250:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2830.890603] LustreError: 95250:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2852.321096] Lustre: 66663:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787363326/real 1787363326] req@00000000f0c96c12 x1874185296307200/t0(0) o400->MGC192.168.204.129@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787363342 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2852.361544] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2855.359256] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2855.364822] LDISKFS-fs (dm-0): recovery complete [ 2855.373570] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2863.585793] LustreError: 66660:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000a128a0f1 x1874185296315776/t0(0) o250->MGC192.168.204.129@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2863.906989] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2865.466983] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2865.475403] Lustre: 95249:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2868.699695] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 2869.226891] Lustre: 95254:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2869.232877] Lustre: 95254:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 2869.248794] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2869.320152] LustreError: 101475:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2869.648265] LustreError: 101475:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2869.977933] LustreError: 101475:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2869.996071] LustreError: 101475:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2870.340317] LustreError: 101475:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2870.365918] LustreError: 101475:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2870.394912] Lustre: 101475:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2870.416110] Lustre: 101475:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 2871.081474] LustreError: 101475:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2871.097230] LustreError: 101475:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2871.416179] LustreError: 101475:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2871.425475] LustreError: 101475:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2872.712448] Lustre: 101475:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2872.742433] Lustre: 101475:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 6 previous similar messages [ 2873.390618] LustreError: 101475:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2873.397993] LustreError: 101475:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2873.720305] LustreError: 101475:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2873.733620] LustreError: 101475:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2876.888948] Lustre: 101475:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2876.895127] Lustre: 101475:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 2877.537197] LustreError: 101475:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2877.544166] LustreError: 101475:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2877.880103] LustreError: 101475:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2877.889325] LustreError: 101475:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2884.916712] Lustre: 101475:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2884.945530] Lustre: 101475:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 24 previous similar messages [ 2885.583251] LustreError: 101475:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2885.591049] LustreError: 101475:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2885.913332] LustreError: 101475:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2885.927595] LustreError: 101475:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2900.936893] Lustre: 101475:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2900.946868] Lustre: 101475:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 49 previous similar messages [ 2901.585859] LustreError: 101475:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2901.592718] LustreError: 101475:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 49 previous similar messages [ 2902.225071] LustreError: 101475:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2902.237885] LustreError: 101475:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2925.080140] Lustre: 101475:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2925.088686] Lustre: 101475:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2925.101374] Lustre: 101475:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client a3d5d6e2-f74c-4920-b58f-5cbf77610133@192.168.204.29@tcp [ 2925.118217] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2925.126380] Lustre: 101475:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@000000003ca92e36 x1874185279311744/t0(8589934771) o101->a3d5d6e2-f74c-4920-b58f-5cbf77610133@192.168.204.29@tcp:586/0 lens 592/0 e 0 to 0 dl 1787363426 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2925.164199] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2925.176656] Lustre: lustre-MDT0000: Denying connection for new client a3d5d6e2-f74c-4920-b58f-5cbf77610133 (at 192.168.204.29@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2925.198286] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2925.249904] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2925.255241] Lustre: Skipped 2 previous similar messages [ 2925.263600] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2925.294194] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:545) [ 2925.297767] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:77 to 0x280000401:577) [ 2940.907316] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2940.917433] Lustre: Skipped 3 previous similar messages [ 2945.521889] Lustre: server umount lustre-OST0000 complete [ 2955.373273] Lustre: server umount lustre-OST0001 complete [ 2961.380650] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2961.381134] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2961.403068] LustreError: Skipped 2 previous similar messages [ 2961.410352] Lustre: Skipped 6 previous similar messages [ 2965.067396] Lustre: server umount lustre-MDT0000 complete [ 2966.497940] LustreError: 95253:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2966.525436] LustreError: 95253:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 51 previous similar messages [ 2969.347743] LustreError: 95235:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787363460 with bad export cookie 16990179954143670150 [ 2969.355552] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2969.361831] LustreError: 95235:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2969.867403] Lustre: server umount lustre-MDT0001 complete [ 2980.007978] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 21:51:09 (1787363469) [ 2988.375644] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2988.390701] Lustre: *** cfs_fail_loc=197, val=0*** [ 3004.064568] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3009.184638] LustreError: 103451:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.129@tcp: failed processing log, type 4: rc = -110 [ 3014.326220] Lustre: *** cfs_fail_loc=197, val=0*** [ 3014.331410] Lustre: Skipped 65 previous similar messages [ 3034.912362] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3034.915908] Lustre: Skipped 3 previous similar messages [ 3040.845936] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3047.715936] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3049.849214] Lustre: server umount lustre-OST0000 complete [ 3058.640066] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 21:52:27 (1787363547) [ 3079.241298] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3083.936559] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3092.405797] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3096.512611] LustreError: 105110:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3096.525061] LustreError: 105110:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3096.539163] LustreError: 105110:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 17, retries 0, failed: rc = -5 [ 3096.889650] Lustre: server umount lustre-MDT0000 complete [ 3127.698277] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [ 3136.860772] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 3169.831335] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3170.134849] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3170.159591] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3170.241769] Lustre: lustre-MDT0000: new disk, initializing [ 3170.331911] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3174.675405] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3184.337254] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3184.424644] Lustre: 109281:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3184.711463] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3188.987961] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3198.029710] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3202.798979] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3209.676501] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3209.915918] Lustre: 111250:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3215.976347] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3217.159864] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3226.372732] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3231.749953] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3231.997297] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3237.170760] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3237.359696] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3242.987588] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3242.995509] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3242.996981] LustreError: Skipped 1 previous similar message [ 3242.998835] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3242.998844] Lustre: Skipped 4 previous similar messages [ 3243.031716] Lustre: Skipped 2 previous similar messages [ 3254.245881] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3254.407461] Lustre: server umount lustre-OST0000 complete [ 3264.196723] Lustre: server umount lustre-MDT0000 complete [ 3267.667104] LustreError: 109274:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787363758 with bad export cookie 16990179954143680741 [ 3267.673087] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3267.676409] LustreError: 109274:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3268.031421] Lustre: server umount lustre-MDT0001 complete [ 3277.089282] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 21:56:05 (1787363765) [ 3287.913220] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 3290.999155] Key type lgssc unregistered [ 3291.271668] LNet: 113743:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3291.275336] LNetError: 113743:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3291.290635] LNet: Removed LNI 192.168.204.129@tcp [ 3292.047156] Key type .llcrypt unregistered [ 3292.049152] Key type ._llcrypt unregistered [ 3309.620639] Key type ._llcrypt registered [ 3309.624665] Key type .llcrypt registered [ 3309.729195] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [ 3325.231974] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 3326.510301] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3326.588980] alg: No test for adler32 (adler32-zlib) [ 3327.650744] Lustre: Lustre: Build Version: 2.17.57_45_g4630995 [ 3328.053325] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 3329.704859] Key type lgssc registered [ 3330.776200] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3371.921661] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3371.950980] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3373.190523] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3373.229589] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3373.341826] Lustre: lustre-MDT0000: new disk, initializing [ 3373.465383] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3373.499991] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3377.484824] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3387.276760] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3387.564816] Lustre: lustre-OST0000: new disk, initializing [ 3387.568996] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3387.575918] Lustre: Skipped 1 previous similar message [ 3387.582848] Lustre: 117918:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3387.679567] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3390.602533] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3390.609223] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3390.641832] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3392.935852] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3402.512478] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3402.807807] Lustre: lustre-OST0001: new disk, initializing [ 3402.817225] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3402.829590] Lustre: 118781:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3402.910103] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3410.115067] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3412.597403] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3412.604976] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3412.631694] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3422.690020] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3422.693155] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3422.710762] Lustre: Skipped 1 previous similar message [ 3422.723330] Lustre: Skipped 1 previous similar message [ 3425.230896] Lustre: server umount lustre-MDT0000 complete [ 3432.163074] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3437.106276] LustreError: 117105:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787363928 with bad export cookie 7111641597827136902 [ 3437.116900] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3437.355608] Lustre: server umount lustre-OST0000 complete [ 3441.294880] Lustre: server umount lustre-OST0001 complete [ 3449.115783] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 21:58:58 (1787363938) [ 3471.035346] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 21:59:20 (1787363960) [ 3479.754138] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 3499.667517] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 21:59:48 (1787363988) [ 3536.517932] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [ 3546.492455] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 3584.686839] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3584.867227] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3584.905714] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3585.012138] Lustre: lustre-MDT0000: new disk, initializing [ 3585.081923] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3585.106370] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3589.455599] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3600.244972] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3600.382757] Lustre: 126543:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3600.434963] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3600.442223] Lustre: Skipped 1 previous similar message [ 3600.513207] Lustre: lustre-MDT0001: new disk, initializing [ 3600.606566] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3600.644070] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3600.651949] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3604.636658] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3613.729275] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3621.124708] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3628.495761] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3628.888946] Lustre: lustre-OST0000: new disk, initializing [ 3628.893196] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3628.907223] Lustre: 128513:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3629.050633] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3636.215742] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3637.321602] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3637.339213] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3637.556074] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3646.765039] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3651.173693] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3651.374700] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3656.505108] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3656.728861] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3663.330823] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3663.339709] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3663.348609] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3663.359863] Lustre: Skipped 1 previous similar message [ 3663.378717] LustreError: Skipped 1 previous similar message [ 3668.462411] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3668.471967] Lustre: Skipped 1 previous similar message [ 3673.057245] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3673.259514] Lustre: server umount lustre-OST0000 complete [ 3677.666970] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3677.674922] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3677.710060] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3677.717626] Lustre: Skipped 1 previous similar message [ 3683.689299] Lustre: server umount lustre-MDT0000 complete [ 3687.460868] LustreError: 127348:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787364178 with bad export cookie 7111641597827138134 [ 3687.482354] LustreError: 127348:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3687.484810] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3687.973386] Lustre: server umount lustre-MDT0001 complete [ 3697.845933] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 22:03:06 (1787364186) [ 3705.437227] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3705.947839] LustreError: 130927:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3705.964956] LustreError: 130927:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 3706.060172] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3710.573368] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3711.462747] LustreError: 130928:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3711.486922] LustreError: 130928:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3715.633851] LustreError: 130927:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3719.096584] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3723.413381] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3731.227371] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3737.440830] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3743.655152] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3744.176123] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3744.181396] Lustre: Skipped 1 previous similar message [ 3751.259610] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3758.420608] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3763.131953] Lustre: *** cfs_fail_loc=159, val=0*** [ 3779.436516] Lustre: lustre-MDT0000: Client b31fd3be-8955-4dd1-a909-5cfb4628445a (at 192.168.204.29@tcp) reconnecting [ 3781.153213] Lustre: *** cfs_fail_loc=159, val=0*** [ 3781.183650] Lustre: Skipped 10 previous similar messages [ 3796.867326] Lustre: lustre-MDT0000: Client b31fd3be-8955-4dd1-a909-5cfb4628445a (at 192.168.204.29@tcp) reconnecting [ 3803.931876] Lustre: server umount lustre-OST0000 complete [ 3811.810308] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3811.815819] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3811.820308] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3811.820316] Lustre: Skipped 2 previous similar messages [ 3811.847496] Lustre: Skipped 2 previous similar messages [ 3818.090832] Lustre: server umount lustre-MDT0000 complete [ 3821.373810] LustreError: 130908:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787364312 with bad export cookie 7111641597827139016 [ 3821.377533] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3821.381333] LustreError: 130908:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3831.993237] Lustre: server umount lustre-MDT0001 complete [ 3844.236847] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 3846.858165] Key type lgssc unregistered [ 3847.131544] LNet: 134712:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3847.140254] LNetError: 134712:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3847.173086] LNet: Removed LNI 192.168.204.129@tcp [ 3848.131896] Key type .llcrypt unregistered [ 3848.133744] Key type ._llcrypt unregistered [ 3877.398328] Key type ._llcrypt registered [ 3877.412525] Key type .llcrypt registered [ 3877.535430] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 3878.750193] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3878.800517] alg: No test for adler32 (adler32-zlib) [ 3879.761732] Lustre: Lustre: Build Version: 2.17.57_45_g4630995 [ 3879.897097] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 3881.504490] Key type lgssc registered [ 3882.483149] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3889.333327] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3889.368697] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3890.763149] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3894.511884] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3900.897748] LustreError: 136129:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3900.917775] LustreError: 136129:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3903.169453] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3903.545950] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3907.913391] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3915.187968] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3921.087444] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3927.699492] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3927.987462] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3934.950211] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3935.198384] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3943.116762] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3952.498271] Lustre: *** cfs_fail_loc=159, val=0*** [ 3968.896634] Lustre: lustre-MDT0000: Client 005908cb-e2d6-498f-a408-609b9cdebfe9 (at 192.168.204.29@tcp) reconnecting [ 3972.634890] Lustre: *** cfs_fail_loc=159, val=0*** [ 3988.854150] Lustre: lustre-MDT0001: Client 005908cb-e2d6-498f-a408-609b9cdebfe9 (at 192.168.204.29@tcp) reconnecting [ 3990.334889] Lustre: *** cfs_fail_loc=159, val=0*** [ 3990.337905] Lustre: Skipped 3 previous similar messages [ 4005.767134] Lustre: lustre-MDT0001: Client 005908cb-e2d6-498f-a408-609b9cdebfe9 (at 192.168.204.29@tcp) reconnecting [ 4022.176783] Lustre: 139187:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4025.225343] Lustre: *** cfs_fail_loc=159, val=0*** [ 4025.229776] Lustre: Skipped 4 previous similar messages [ 4040.517704] Lustre: lustre-MDT0000: Client 4fc4cf26-2ab5-44d9-a17a-ee25a9cc0b0e (at 192.168.204.29@tcp) reconnecting [ 4042.278982] Lustre: *** cfs_fail_loc=159, val=0*** [ 4042.295534] Lustre: Skipped 15 previous similar messages [ 4057.962988] Lustre: lustre-MDT0000: Client 4fc4cf26-2ab5-44d9-a17a-ee25a9cc0b0e (at 192.168.204.29@tcp) reconnecting [ 4059.251737] Lustre: 139477:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4066.284094] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4066.304137] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4071.395260] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4071.404589] Lustre: Skipped 2 previous similar messages [ 4071.756560] Lustre: server umount lustre-OST0000 complete [ 4076.513616] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4076.534807] Lustre: Skipped 1 previous similar message [ 4076.544052] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4076.550670] Lustre: Skipped 1 previous similar message [ 4081.647058] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4081.654211] Lustre: Skipped 1 previous similar message [ 4082.142485] Lustre: server umount lustre-MDT0000 complete [ 4085.791668] LustreError: 136111:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787364576 with bad export cookie 10769684913622254977 [ 4085.801774] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4085.811092] LustreError: 136111:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4086.101473] Lustre: server umount lustre-MDT0001 complete [ 4098.042732] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 4100.802082] Key type lgssc unregistered [ 4101.075991] LNet: 140539:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4101.081347] LNetError: 140539:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4101.100817] LNet: Removed LNI 192.168.204.129@tcp [ 4101.883704] Key type .llcrypt unregistered [ 4101.886746] Key type ._llcrypt unregistered [ 4131.846520] Key type ._llcrypt registered [ 4131.850838] Key type .llcrypt registered [ 4132.010661] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 4133.456105] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4133.580626] alg: No test for adler32 (adler32-zlib) [ 4134.957288] Lustre: Lustre: Build Version: 2.17.57_45_g4630995 [ 4135.352309] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 4137.160199] Key type lgssc registered [ 4138.902536] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4148.444959] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4148.462209] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4149.934591] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4154.863427] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4159.458833] LustreError: 141956:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4159.476187] LustreError: 141956:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4164.583329] LustreError: 141955:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4166.080669] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4167.040059] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4173.929580] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4184.454551] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4191.500956] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4199.026853] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4199.529817] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4206.557638] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4211.836591] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4215.150755] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4220.030344] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4225.787277] Lustre: 144542:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4230.015884] Lustre: 144590:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4232.102359] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4233.260744] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4238.636954] Lustre: 144737:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4239.844795] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4241.078691] Lustre: 144836:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4245.985556] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4246.000451] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4246.024054] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4248.034481] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4248.039547] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4248.053877] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4252.207184] Lustre: server umount lustre-OST0000 complete [ 4258.786092] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4258.807380] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4258.808964] Lustre: Skipped 1 previous similar message [ 4258.834136] Lustre: Skipped 2 previous similar messages [ 4263.136605] Lustre: server umount lustre-MDT0000 complete [ 4263.905706] LustreError: 142683:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4263.933491] LustreError: 142683:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4267.236577] LustreError: 141936:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787364758 with bad export cookie 7891048228239470717 [ 4267.252281] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4267.257883] LustreError: 141936:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4267.556510] Lustre: server umount lustre-MDT0001 complete [ 4280.966028] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 4284.335622] Key type lgssc unregistered [ 4284.652914] LNet: 145900:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4284.673207] LNetError: 145900:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4284.696122] LNet: Removed LNI 192.168.204.129@tcp [ 4285.755719] Key type .llcrypt unregistered [ 4285.760095] Key type ._llcrypt unregistered [ 4315.868468] Key type ._llcrypt registered [ 4315.870776] Key type .llcrypt registered [ 4315.965822] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 4317.174168] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4317.267034] alg: No test for adler32 (adler32-zlib) [ 4318.325231] Lustre: Lustre: Build Version: 2.17.57_45_g4630995 [ 4318.630420] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 4320.353536] Key type lgssc registered [ 4321.682499] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4331.149495] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4331.164403] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4332.953184] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4339.881842] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4342.246946] LustreError: 147316:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4342.261970] LustreError: 147316:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4347.370455] LustreError: 147317:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4349.567815] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4350.150590] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4355.925571] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4365.032487] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4372.630308] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4380.284715] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4380.711335] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4388.672423] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4395.095517] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4398.721543] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4403.310334] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4411.050196] Lustre: *** cfs_fail_loc=159, val=0*** [ 4426.623670] Lustre: lustre-MDT0000: Client 187b8696-e296-45f6-9d7b-ed78fddb461f (at 192.168.204.29@tcp) reconnecting [ 4444.641867] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4444.653116] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4444.656667] Lustre: Skipped 1 previous similar message [ 4444.672599] Lustre: Skipped 1 previous similar message [ 4449.768033] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4449.779803] Lustre: Skipped 1 previous similar message [ 4450.728630] Lustre: server umount lustre-OST0000 complete [ 4457.953175] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4457.960575] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4457.977322] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4460.001942] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4460.003247] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4460.036958] Lustre: Skipped 1 previous similar message [ 4461.741171] Lustre: server umount lustre-MDT0000 complete [ 4465.128725] LustreError: 147317:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4465.156292] LustreError: 147317:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4465.867758] LustreError: 149943:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787364956 with bad export cookie 17448791336809551087 [ 4465.870321] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4465.884798] LustreError: 149943:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4466.414840] Lustre: server umount lustre-MDT0001 complete [ 4479.216211] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 4482.825542] Key type lgssc unregistered [ 4483.209192] LNet: 151001:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4483.217738] LNetError: 151001:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4483.236588] LNet: Removed LNI 192.168.204.129@tcp [ 4484.070981] Key type .llcrypt unregistered [ 4484.073651] Key type ._llcrypt unregistered [ 4511.709094] Key type ._llcrypt registered [ 4511.711249] Key type .llcrypt registered [ 4511.865577] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 4512.645987] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4512.772124] alg: No test for adler32 (adler32-zlib) [ 4513.961872] Lustre: Lustre: Build Version: 2.17.57_45_g4630995 [ 4514.278047] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 4516.064798] Key type lgssc registered [ 4517.283304] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4529.120923] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4529.172189] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4531.013394] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4537.196795] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4540.389739] LustreError: 152417:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4540.419791] LustreError: 152417:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4545.527121] LustreError: 152416:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4548.500445] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4548.872380] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4554.344249] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4563.316744] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4570.804707] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4580.140360] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4580.740814] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4588.443118] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4594.052984] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4597.067823] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4600.239805] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4621.113396] Lustre: lustre-MDT0001: evicting b4549288-296b-4565-9213-be9869c052d0 (at 192.168.204.29@tcp) by administrative request [ 4621.125468] Lustre: Skipped 1 previous similar message [ 4650.476094] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4650.489451] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4650.517055] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4651.513505] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4651.524535] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4651.526462] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4656.585497] Lustre: server umount lustre-OST0000 complete [ 4661.739113] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4661.746502] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4661.771212] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4661.780583] Lustre: Skipped 2 previous similar messages [ 4666.853250] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4666.874622] Lustre: Skipped 1 previous similar message [ 4667.481585] Lustre: server umount lustre-MDT0000 complete [ 4671.486632] LustreError: 152397:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787365162 with bad export cookie 6010339608029909648 [ 4671.517272] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4671.912313] Lustre: server umount lustre-MDT0001 complete [ 4685.259471] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 4689.490824] Key type lgssc unregistered [ 4689.865977] LNet: 156495:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4689.872568] LNetError: 156495:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4690.920035] LNet: Removed LNI 192.168.204.129@tcp [ 4692.117053] Key type .llcrypt unregistered [ 4692.120226] Key type ._llcrypt unregistered [ 4772.706690] Key type ._llcrypt registered [ 4772.708910] Key type .llcrypt registered [ 4772.822846] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 4774.284424] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4774.327650] alg: No test for adler32 (adler32-zlib) [ 4775.567774] Lustre: Lustre: Build Version: 2.17.57_45_g4630995 [ 4775.941551] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 4777.704483] Key type lgssc registered [ 4779.172452] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4788.094505] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4788.129290] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4789.744710] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4794.643370] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4799.983131] LustreError: 158988:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4800.019774] LustreError: 158988:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4804.958030] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4805.092814] LustreError: 158987:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4805.530488] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4810.635815] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4818.748918] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4825.121900] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4833.578357] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4833.924481] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4834.951528] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4840.913680] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4849.604781] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4850.371940] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4860.897885] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4860.901304] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4860.909471] LustreError: Skipped 1 previous similar message [ 4860.910358] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4860.937741] Lustre: Skipped 1 previous similar message [ 4865.415549] Lustre: server umount lustre-OST0000 complete [ 4871.149229] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4871.156077] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4871.177855] Lustre: Skipped 1 previous similar message [ 4871.196033] Lustre: Skipped 2 previous similar messages [ 4876.005297] Lustre: server umount lustre-MDT0000 complete [ 4876.257724] LustreError: 158988:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4876.265784] LustreError: 158988:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4880.188792] LustreError: 158968:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787365371 with bad export cookie 6359908910233524448 [ 4880.216982] LustreError: 158968:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4880.219703] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4880.532356] Lustre: server umount lustre-MDT0001 complete [ 4892.098684] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 4895.253483] Key type lgssc unregistered [ 4895.634802] LNet: 162542:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4895.642213] LNetError: 162542:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4895.671385] LNet: Removed LNI 192.168.204.129@tcp [ 4896.295828] Key type .llcrypt unregistered [ 4896.297561] Key type ._llcrypt unregistered [ 4927.725837] Key type ._llcrypt registered [ 4927.727375] Key type .llcrypt registered [ 4927.994554] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 4929.652813] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4929.667589] alg: No test for adler32 (adler32-zlib) [ 4930.716669] Lustre: Lustre: Build Version: 2.17.57_45_g4630995 [ 4930.997467] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 4932.712303] Key type lgssc registered [ 4934.328826] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4950.534915] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 22:23:59 (1787365439) [ 4967.260503] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [ 4977.990430] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 5023.093167] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5023.144052] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5024.490878] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5024.538849] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5024.655489] Lustre: lustre-MDT0000: new disk, initializing [ 5024.705390] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5024.716763] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5029.558882] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 5040.767113] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5040.946845] Lustre: 166809:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5041.002880] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5041.008629] Lustre: Skipped 1 previous similar message [ 5041.203520] Lustre: lustre-MDT0001: new disk, initializing [ 5041.305235] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5041.358863] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5041.368572] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5046.627589] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 5058.062494] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5064.881226] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5072.993920] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5073.524466] Lustre: lustre-OST0000: new disk, initializing [ 5073.527844] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5073.545325] Lustre: 168780:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5073.702526] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5079.106564] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5079.126051] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5079.273613] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5081.338224] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 5092.587649] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5105.121515] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5105.129938] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5105.135326] LustreError: Skipped 1 previous similar message [ 5105.147177] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5109.903882] Lustre: server umount lustre-OST0000 complete [ 5115.369036] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5115.381654] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5115.396076] Lustre: Skipped 2 previous similar messages [ 5115.398654] Lustre: Skipped 1 previous similar message [ 5120.610145] Lustre: server umount lustre-MDT0000 complete [ 5124.557217] LustreError: 167604:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787365615 with bad export cookie 18041939321784443245 [ 5124.560975] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5124.573906] LustreError: 167604:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5124.966478] Lustre: server umount lustre-MDT0001 complete [ 5137.600943] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 5140.513299] Key type lgssc unregistered [ 5140.865511] LNet: 170688:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5140.871848] LNetError: 170688:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5140.884425] LNet: Removed LNI 192.168.204.129@tcp [ 5141.782816] Key type .llcrypt unregistered [ 5141.786051] Key type ._llcrypt unregistered [ 5167.542300] Key type ._llcrypt registered [ 5167.547218] Key type .llcrypt registered [ 5167.710891] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 5169.104464] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5169.139760] alg: No test for adler32 (adler32-zlib) [ 5170.636672] Lustre: Lustre: Build Version: 2.17.57_45_g4630995 [ 5170.821891] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 5172.472674] Key type lgssc registered [ 5173.629805] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5183.516562] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5183.536693] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5185.245925] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5190.525829] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 5195.751586] LustreError: 172102:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5195.790052] LustreError: 172102:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5200.874814] LustreError: 172103:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5201.727914] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5202.146990] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5207.131000] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 5216.065791] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5222.681379] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5230.258681] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5230.709966] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5231.740377] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5236.950683] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 5245.438426] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5253.020157] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5253.039140] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5253.063860] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5253.074136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.094236] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5254.270843] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5254.306928] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5254.330375] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5254.345847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.375945] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5255.367436] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5255.383272] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5255.391865] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5255.396599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.404127] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5256.296050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.267752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.134506] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5258.142368] Lustre: Skipped 2 previous similar messages [ 5258.162152] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5258.181535] Lustre: Skipped 2 previous similar messages [ 5258.185806] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5258.197260] Lustre: Skipped 2 previous similar messages [ 5258.208435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.234806] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5258.243258] Lustre: Skipped 2 previous similar messages [ 5259.153678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.886154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.778266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.749247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.826164] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5262.834333] Lustre: Skipped 4 previous similar messages [ 5262.848439] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5262.859673] Lustre: Skipped 4 previous similar messages [ 5262.865504] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5262.868586] Lustre: Skipped 4 previous similar messages [ 5262.874545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.880877] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5262.890217] Lustre: Skipped 4 previous similar messages [ 5263.656688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.465640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.250483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.062335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.863398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.668889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.416960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.381485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.274676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.165302] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5271.172606] Lustre: Skipped 9 previous similar messages [ 5271.187762] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5271.198956] Lustre: Skipped 9 previous similar messages [ 5271.204610] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5271.210391] Lustre: Skipped 9 previous similar messages [ 5271.213952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.224893] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5271.228376] Lustre: Skipped 9 previous similar messages [ 5272.016216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.019556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.945441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.852307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.634633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.531943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.412396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.476134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.356352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.192885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.322602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.237686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.265628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.169521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.021967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.049058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.139891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.249555] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5288.261384] Lustre: Skipped 17 previous similar messages [ 5288.294752] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5288.320074] Lustre: Skipped 17 previous similar messages [ 5288.341760] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5288.350823] Lustre: Skipped 17 previous similar messages [ 5288.363981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.388892] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5288.398736] Lustre: Skipped 17 previous similar messages [ 5289.554914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.671418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.845424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.847818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.907898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.741036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.543468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.422448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.469422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.293662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.096289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.922234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.766528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.786828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.727932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.587648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.533430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.631452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.684361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.787943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.930160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.036464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.939839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.917544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.976933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.012859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.089773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.110661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.121124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.318735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.361223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.434427] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5320.449105] Lustre: Skipped 31 previous similar messages [ 5320.465923] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5320.486454] Lustre: Skipped 31 previous similar messages [ 5320.495614] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5320.506760] Lustre: Skipped 31 previous similar messages [ 5320.512909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.533431] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5320.551865] Lustre: Skipped 31 previous similar messages [ 5321.501672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.603506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.473671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.572284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.481642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.487585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.553216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.655831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.599404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.593655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.523379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.661613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.820441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.908976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.093891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.149942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.225906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.148451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.009708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.174994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.428512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.676643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.512902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.480891] LustreError: 172123:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5345.496132] LustreError: 172123:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5345.496418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.592886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.763880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.851226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.929503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.153604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.086648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.222527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.132391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.112872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.017416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.892076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.877734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.063727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.953489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.058465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.103239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.479303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.587182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.518893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.828406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.955183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.069429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.137302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.092958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.048884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.018537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.152427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.041523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.990820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.910862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.738703] LustreError: 172123:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5377.746507] LustreError: 172123:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5377.748420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.598869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.460673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.425771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.246696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.375057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.145275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.020031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.981934] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5384.984975] Lustre: Skipped 62 previous similar messages [ 5384.998165] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5385.004925] Lustre: Skipped 62 previous similar messages [ 5385.007929] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5385.011314] Lustre: Skipped 62 previous similar messages [ 5385.014113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.021463] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5385.025060] Lustre: Skipped 62 previous similar messages [ 5386.010617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.937863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.981027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.760404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.471764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.405325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.296567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.403905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.434041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.366246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.239926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.307563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.212957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.527801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.373438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.603842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.752579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.832476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.112215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.348379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.599775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.590223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.952990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.192903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.444632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.492702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.738691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.896533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.854032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.953808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.009069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.224195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.390841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.863621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.137357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.107392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.417805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.608877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.655828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.690340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.769662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.577590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.371818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.281134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.105056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.012479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.056712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.987520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.974523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.903887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.858980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.964033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.162231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.330906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.417897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.290227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.329269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.207397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.114610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.044104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.923532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.875533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.912967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.925786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.146480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.294106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.345122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.319491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.307376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.400853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.337782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.587515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.640108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.507365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.528247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.531500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.453246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.459503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.328166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.314907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.381277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.293263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.221960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.389401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.473540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.356867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.292570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.220910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.300357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.267095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.248806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.209749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.245653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.490201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.671337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.683895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.072598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.200375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.422283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.340952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.281494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.252836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.606437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.540966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.541949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.536136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.507271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.393843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.298254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.113274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.027531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.945170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.940787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.866249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.677153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.643088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.592288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.514952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.435600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.361430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.354846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.321195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.366742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.417924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.438482] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5513.443685] Lustre: Skipped 124 previous similar messages [ 5513.462722] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5513.472470] Lustre: Skipped 124 previous similar messages [ 5513.478309] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5513.482498] Lustre: Skipped 124 previous similar messages [ 5513.487154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.505338] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5513.517217] Lustre: Skipped 124 previous similar messages [ 5514.467848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.448686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.421858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.159088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.933617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.927267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.963283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.818794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.846015] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.786429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.754889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.636967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.604045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.525567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.487198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.451484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.528357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.456740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.541283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.591674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.657479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.666487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.534524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.493758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.311632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.203516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.006662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.983973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.920506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.840312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.702861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.866455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.827804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.741432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.851269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.140281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.340756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.379157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.318307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.180114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.196645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.260588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.202554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.198263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.084330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.194156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.230906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.278573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.384231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.357801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.438552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.425741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.521503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.510135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.437359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.434079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.481356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.549349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.526981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.572231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.465138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.489804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.483853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.493664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.505584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.612258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.589299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.707751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.555654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.565666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.611320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.450054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.319209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.210274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.066150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.912224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.729462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.604541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.609064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.721766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.564625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.712516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.808205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.842756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.704590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.679810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.624551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.598396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.599632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.453508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.227373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.945286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.829607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.634588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.791412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.689694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.697935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.631962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.597877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.507878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.414859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.446429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.394235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.365909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.500781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.605900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.622227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.625986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.478791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.405626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.312546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.362057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.202758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.023429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.789290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.742472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.683354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.580422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.426187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.294570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.296312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.601405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.579691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.548322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.315512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.287152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.199062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.156558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.028844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.974630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.858530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.928268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.827137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.801504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.678743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.687910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.572529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.555585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.469665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.346400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.334465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.227669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.216172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.279491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.184607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.007187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.773678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.727578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.822187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.897956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.900831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.814727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.719587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.528606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.458319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.489817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.328414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.324579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.447190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.523206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.596783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.691579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.828375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.732103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.793573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.811331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.754162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.602946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.836722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.762496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.712378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.787771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.722547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.686155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.601719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.615265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.445252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.440844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.381888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.492254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.467075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.413509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.448800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.370865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.397921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.596500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.825046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.768231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.723683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.792254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.839867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.671425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.666440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.648382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.548971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.417220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.324692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.191847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.017563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.856771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.758590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.711664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.696358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.696495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.611408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.410227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.227299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.213275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.127292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.144787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.098889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.039981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.917222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.057905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.918149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.872697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.903826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.020307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.024629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.024571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.889226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.749408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.677443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.593726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.564613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.499494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.286375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.163381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.976410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.811907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.736506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.598667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.530574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.421855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.278625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.933388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.799666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.479965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.370879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.074704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.828652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.756139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.506132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.369028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.174295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.413249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.472431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.364887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.221472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.949272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.544813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.538276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.490664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.471538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.531347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.436888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.497961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.376877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.358309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.167515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.036913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.067613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.001658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.966225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.814670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.892031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.828961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.770874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.659515] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5769.663275] Lustre: Skipped 268 previous similar messages [ 5769.673379] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5769.682749] Lustre: Skipped 268 previous similar messages [ 5769.689349] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5769.697528] Lustre: Skipped 268 previous similar messages [ 5769.701529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.713212] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5769.720528] Lustre: Skipped 268 previous similar messages [ 5770.561047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.456409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.611325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.704956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.684571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.780401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.645815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.551278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.439367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.321437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.013566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.927699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.851595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.850668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.968735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.882681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.886377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.972811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.950365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.841907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.750951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.776170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.843822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.776158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.853852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.937550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.055031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.039841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.958334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.906447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.837338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.890816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.001709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.944958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.948563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.794324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.718529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.661995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.708691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.555838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.609856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.648885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.667438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.627627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.565525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.530400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.614293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.536976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.583149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.693214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.535509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.629543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.694120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.619577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.578774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.629749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.600500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.638803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.527933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.535191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.564194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.570867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.598995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.549604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.580758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.635484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.690409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.669750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.567767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.505501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.456166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.416257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.406046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.216043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.229364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.140169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.951388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.047320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.964638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.985150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.882554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.790831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.823887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.810714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.708431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.511085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.237981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.076723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.943789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.830304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.688566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.685336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.665615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.657065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.529164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.659396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.621726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.576541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.414989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.422148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.270994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.405301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.544637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.383662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.236980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.107593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.187776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.218402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.162486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.110873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.134649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.097244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.248415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.353832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.495248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.461736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.542636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.601567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.533510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.573264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.747791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.945371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.290613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.437018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.406546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.033066] LustreError: 172123:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5893.042929] LustreError: 172123:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5893.054955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.826492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.723510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.842634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.088662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.263227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.178484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.258054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.245511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.658794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.874478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.018576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.000915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.157911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.327679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.292952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.248690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.202196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.359842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.387953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.868682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.851494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.796316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.715161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.604119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.431731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.187169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.059722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.801069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.125966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.956084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.725778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.562665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.137095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.417211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.160556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.001678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.850375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.809280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.620765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.504068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.114653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.021923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.831221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.529747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.155350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.779512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.518783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.344814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.101917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.109510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.063611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.871716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.819346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.847411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.785324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.739354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.924057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.818047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.723180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.617445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.579668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.494768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.476962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.400322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.304970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.042494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.903721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.709827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.650789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.424345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.275942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.127150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.017935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.922385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.737746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.594771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.393734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.201915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.931445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.696855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.414537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.133563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.845392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.625710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.389957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.230156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.025858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.801491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.617332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.448503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.239589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.015222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.856744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.635229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.308343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.063373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.980052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.774226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.609477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.411063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.196782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.981237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.777569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.667664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.532160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.370512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.060540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.987675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.803658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.764637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.570403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.444472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.647056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.662601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.871633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.833956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.014143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.848852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.767771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.718648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.618914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.406030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.219240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.000273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.979241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.954640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.874498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.711852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.704494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.688934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.549686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.449733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.322862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.141241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.947026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.891863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.942889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.024509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.951577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.031663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.919465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.914159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.906160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.897558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.705324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.655520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.532396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.568367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.484058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.481512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.621701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.853278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.760879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.751403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.841577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.006723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.041828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.209807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.372065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.600830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.609310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.708269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.805788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.918737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.059872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.080694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.243566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.134127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.952542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.914883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.899810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.715201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.582955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.394515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.279354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.248691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.204974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.080657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.985800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.897941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.889054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.711440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.646211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.457679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.311381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.265975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.320419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.393613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.342621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.303392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.310221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.504717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.489734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.491925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.534459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.340334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.488792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.601966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.641978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.735138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.664269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.756192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.708500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.807649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.842855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.922567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.891487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.822213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.688312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.609767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.482488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.537030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.611385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.467975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.313658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.211706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.232430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.354886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.321989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.298915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.324713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.316482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.346508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.227573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.279090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.170358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.220584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.102062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.053822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.989554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.977717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.776965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.651205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.481337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.402985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.356565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.486262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.618398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.693862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.557925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.503685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.510345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.501531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.494708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.375624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.350578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.478193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.436755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.464664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.390789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.352786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.351786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.399785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.501070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.420673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.554375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.572631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.509925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.402180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.496532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.440249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.416429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.492685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.671573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.695371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.771294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.875988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.098166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.255029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.328682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.257098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.204307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.193550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.016483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.848228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.693809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.862288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.831081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.728910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.627210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.547240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.421387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.419696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.358166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.357510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.240653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.238937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.278443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.265263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.157334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.119380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.886274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.933693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.977881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.032548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.187152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.514723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.718111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.733289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.667024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.583482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.546482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.462308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.372227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.314326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.305302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.457032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.648487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.554729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.696271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.922532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.055938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.149586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.158438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.264438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.356069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.287831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.187113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.087808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.244445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.293396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.382994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.483703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.514594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.513209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.483063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.747753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.722632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.530929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.403101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.326182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.123175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.975124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.892738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.912173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.835258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.930756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.974577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.981836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.924314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.761319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.710601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.642849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.654289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.526410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.407246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6667.929727] LustreError: 174008:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6667.945406] LustreError: 172123:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7425.004279] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 7425.005320] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7425.013351] LustreError: Skipped 1 previous similar message [ 7425.023024] Lustre: Skipped 472 previous similar messages [ 7425.027079] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7430.628100] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7430.636390] Lustre: Skipped 1 previous similar message [ 7430.847776] Lustre: server umount lustre-OST0000 complete [ 7435.749255] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7435.755936] Lustre: Skipped 1 previous similar message [ 7440.865916] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7440.881563] Lustre: Skipped 1 previous similar message [ 7441.350884] Lustre: server umount lustre-MDT0000 complete [ 7445.104976] LustreError: 172084:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787367936 with bad export cookie 5913019073777528214 [ 7445.119769] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7445.131957] LustreError: 172084:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7445.496339] Lustre: server umount lustre-MDT0001 complete [ 7456.457559] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 7458.910412] Key type lgssc unregistered [ 7459.196710] LNet: 227400:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7459.204323] LNetError: 227400:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7459.225530] LNet: Removed LNI 192.168.204.129@tcp [ 7460.076564] Key type .llcrypt unregistered [ 7460.079810] Key type ._llcrypt unregistered [ 7488.156774] Key type ._llcrypt registered [ 7488.159727] Key type .llcrypt registered [ 7488.350352] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 7491.344092] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7491.384155] alg: No test for adler32 (adler32-zlib) [ 7492.631036] Lustre: Lustre: Build Version: 2.17.57_45_g4630995 [ 7493.150021] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 7494.928189] Key type lgssc registered [ 7496.544980] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7505.948235] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7505.971482] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7507.397719] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7511.997860] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 7517.670058] LustreError: 228817:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7517.684206] LustreError: 228817:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7521.418587] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7521.881542] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7526.538947] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 7535.287718] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7542.548851] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7550.721574] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7551.106647] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7552.134715] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7559.366215] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 7568.218873] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7574.196801] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7576.132819] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7576.149280] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7576.156318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7576.167378] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7576.173215] LustreError: 228836:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7581.525555] Lustre: lustre-OST0000: Not available for connect from 192.168.204.29@tcp (stopping) [ 7582.177384] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7582.190482] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7582.211620] Lustre: Skipped 1 previous similar message [ 7582.214653] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7582.251841] Lustre: Skipped 1 previous similar message [ 7584.604550] Lustre: server umount lustre-OST0000 complete [ 7596.241075] LustreError: 228836:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7619.296405] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7619.711112] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7622.765230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7622.785074] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7622.789435] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7627.118416] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 7635.709466] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7637.635456] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7644.014439] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff91e5505ce000.ost_server_uuid 50 [ 7645.752352] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff91e5505ce000.ost_server_uuid in IDLE state after 0 sec [ 7651.301805] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7651.312269] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7651.340087] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7653.861933] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7653.862893] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7653.878522] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7656.386883] Lustre: server umount lustre-OST0000 complete [ 7664.108154] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7664.129826] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7664.134196] Lustre: Skipped 2 previous similar messages [ 7666.761272] Lustre: server umount lustre-MDT0000 complete [ 7669.219690] LustreError: 228817:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7669.244683] LustreError: 228817:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7670.509731] LustreError: 228798:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787368161 with bad export cookie 4672327085441325396 [ 7670.514839] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7670.518337] LustreError: 228798:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7670.808321] Lustre: server umount lustre-MDT0001 complete [ 7683.569676] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 7686.277488] Key type lgssc unregistered [ 7686.521295] LNet: 234034:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7686.531843] LNetError: 234034:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7686.552827] LNet: Removed LNI 192.168.204.129@tcp [ 7687.477180] Key type .llcrypt unregistered [ 7687.479363] Key type ._llcrypt unregistered [ 7721.438637] Key type ._llcrypt registered [ 7721.445318] Key type .llcrypt registered [ 7721.588629] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 7722.587903] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7722.617907] alg: No test for adler32 (adler32-zlib) [ 7723.698735] Lustre: Lustre: Build Version: 2.17.57_45_g4630995 [ 7723.991911] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 7725.688132] Key type lgssc registered [ 7726.915565] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7738.974212] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7739.022255] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7740.441193] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7746.098484] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 7750.629939] LustreError: 235766:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7750.647966] LustreError: 235766:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7755.749038] LustreError: 235767:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7756.543484] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7757.088036] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7761.804295] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 7764.895905] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7771.176938] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7771.417377] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7778.967775] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 7785.775088] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7788.068586] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7788.199969] Lustre: lustre-OST0001: new disk, initializing [ 7788.204568] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7788.210171] Lustre: 238075:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7788.357365] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7794.343797] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7794.358513] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7794.476740] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7794.949360] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 7805.829412] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7812.636458] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7828.962280] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7828.968469] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7828.979873] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7830.497947] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7830.501451] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7830.506441] Lustre: Skipped 1 previous similar message [ 7832.509313] Lustre: server umount lustre-MDT0000 complete [ 7835.625489] LustreError: 235766:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7835.635537] LustreError: 235766:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 7835.925059] LustreError: 235748:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787368326 with bad export cookie 12737360739050366585 [ 7835.927648] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7835.936070] LustreError: 235748:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7836.259386] Lustre: server umount lustre-MDT0001 complete [ 7851.175794] Lustre: server umount lustre-OST0000 complete [ 7865.605490] Lustre: server umount lustre-OST0001 complete [ 7885.499841] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 7895.411552] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7895.782765] LustreError: 241186:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7895.863487] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7900.491032] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 7901.157091] LustreError: 241187:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7909.494638] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7909.940971] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7915.159765] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 7918.516477] Lustre: 242328:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7925.149064] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7932.852607] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 7934.763116] LustreError: 242684:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7934.777919] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:129) [ 7934.784607] LustreError: 242684:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7941.178094] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 7943.118689] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7943.455898] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7943.461212] Lustre: Skipped 1 previous similar message [ 7948.792340] Lustre: mylustre-OST0001: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:33) [ 7950.249686] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 7958.484502] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7962.325720] Lustre: 244200:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7974.378088] Lustre: mylustre-MDT0000-lwp-OST0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7974.387464] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7974.395040] Lustre: Skipped 3 previous similar messages [ 7974.411889] Lustre: Skipped 6 previous similar messages [ 7979.498362] LustreError: 242152:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7979.517335] LustreError: 242152:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 7979.566150] Lustre: server umount mylustre-MDT0000 complete [ 7983.500540] LustreError: 241161:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787368474 with bad export cookie 12737360739050369448 [ 7983.524077] LustreError: 241161:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7983.525878] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7983.964173] Lustre: server umount mylustre-MDT0001 complete [ 7998.193120] Lustre: server umount mylustre-OST0000 complete [ 8000.801903] Lustre: 235028:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787368475/real 1787368475] req@ffff96772122f480 x1874191412524672/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787368491 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8000.833795] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8000.843022] Lustre: Skipped 1 previous similar message [ 8002.258921] Lustre: server umount mylustre-OST0001 complete [ 8025.009909] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 8035.547277] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8035.927894] LustreError: 246510:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8036.015826] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8039.685452] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 8050.031151] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8055.472797] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 8058.787458] Lustre: 247653:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8067.481351] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8075.880861] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 8075.978232] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:161) [ 8080.116314] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 8085.793530] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8091.656059] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:65) [ 8092.891341] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 8101.587422] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8105.823821] Lustre: 249524:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8117.218374] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8117.228222] Lustre: tfs-MDT0000-lwp-OST0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8117.244321] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 8117.258722] Lustre: Skipped 2 previous similar messages [ 8117.273180] Lustre: Skipped 3 previous similar messages [ 8119.465552] Lustre: server umount tfs-MDT0000 complete [ 8122.353108] LustreError: 246511:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8122.384846] LustreError: 246511:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 10 previous similar messages [ 8124.078693] LustreError: 246487:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787368615 with bad export cookie 12737360739050371625 [ 8124.092129] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8124.098456] LustreError: 246487:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8124.501084] Lustre: server umount tfs-MDT0001 complete [ 8138.513642] Lustre: server umount tfs-OST0000 complete [ 8153.597200] Lustre: server umount tfs-OST0001 complete [ 8180.540904] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 8193.135401] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8193.896331] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8193.911315] Lustre: Skipped 3 previous similar messages [ 8199.491642] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 8208.799560] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8213.222401] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 8216.605153] Lustre: 252919:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8223.889449] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8231.383211] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 8233.684569] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:193) [ 8238.866269] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 8240.127079] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8245.763836] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:97) [ 8246.611190] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 8253.648416] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8257.156454] Lustre: 254794:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8263.038302] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 23:19:12 (1787368752) [ 8280.545589] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8280.556561] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8280.569758] Lustre: Skipped 1 previous similar message [ 8280.574987] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8284.444225] Lustre: server umount lustre-MDT0000 complete [ 8286.698104] LustreError: 251779:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8286.727444] LustreError: 251779:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 10 previous similar messages [ 8287.812213] LustreError: 251755:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787368778 with bad export cookie 12737360739050373543 [ 8287.813614] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8287.827683] LustreError: 251755:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8291.817084] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8291.824932] Lustre: Skipped 5 previous similar messages [ 8294.311641] Lustre: server umount lustre-MDT0001 complete [ 8312.288145] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8312.527573] Lustre: server umount lustre-OST0000 complete [ 8323.761581] Lustre: server umount lustre-OST0001 complete [ 8330.519682] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [ 8342.513081] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 8384.016458] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8384.265681] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8384.316894] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8384.412432] Lustre: lustre-MDT0000: new disk, initializing [ 8384.534869] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 8384.543380] Lustre: Skipped 3 previous similar messages [ 8384.566379] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8390.125805] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 8401.373933] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8401.454278] Lustre: 257981:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8401.520375] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8401.524322] Lustre: Skipped 1 previous similar message [ 8401.622228] Lustre: lustre-MDT0001: new disk, initializing [ 8401.724208] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8401.733870] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8406.545360] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 8417.202892] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8417.546490] Lustre: lustre-OST0000: new disk, initializing [ 8417.549608] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8417.556193] Lustre: 259561:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8419.140169] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8419.147726] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8419.242356] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8424.059036] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 8435.042735] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8435.274836] Lustre: lustre-OST0001: new disk, initializing [ 8435.280308] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8435.288368] Lustre: 260433:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8436.544949] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8436.557951] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8436.670130] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8442.624083] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 8455.919407] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 23:22:25 (1787368945) [ 8472.877516] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 23:22:41 (1787368961) [ 8490.980385] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8490.993364] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8491.002060] Lustre: Skipped 5 previous similar messages [ 8491.008255] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8491.012143] Lustre: Skipped 3 previous similar messages [ 8494.286534] Lustre: server umount lustre-OST0000 complete [ 8504.016845] Lustre: server umount lustre-MDT0000 complete [ 8507.408785] LustreError: 257972:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787368998 with bad export cookie 12737360739050375321 [ 8507.416183] LustreError: 257972:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8507.417064] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8513.775046] Lustre: server umount lustre-MDT0001 complete [ 8524.792782] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 8597.037928] Lustre: server umount lustre-OST0001 complete [ 8603.654293] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [ 8620.417974] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 8661.014094] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8661.237375] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8661.264649] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8661.323326] Lustre: lustre-MDT0000: new disk, initializing [ 8661.398581] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8661.408829] Lustre: Skipped 3 previous similar messages [ 8661.440230] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8666.427972] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 8677.133735] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8677.237485] Lustre: 265276:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8677.298571] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8677.300752] Lustre: Skipped 1 previous similar message [ 8677.361936] Lustre: lustre-MDT0001: new disk, initializing [ 8677.546671] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8677.581096] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8682.879967] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 8692.911911] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8700.075511] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8706.170493] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8706.625363] Lustre: 267248:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8708.108484] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8708.376748] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8713.177078] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 8723.381194] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8733.683863] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8733.701419] LustreError: Skipped 1 previous similar message [ 8733.705345] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8733.727118] Lustre: Skipped 5 previous similar messages [ 8733.730040] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8733.732374] Lustre: Skipped 10 previous similar messages [ 8738.651411] Lustre: server umount lustre-OST0000 complete [ 8743.906709] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8743.915520] LustreError: Skipped 1 previous similar message [ 8757.216162] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8757.657396] Lustre: server umount lustre-MDT0000 complete [ 8759.779407] LustreError: 265282:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8759.806302] LustreError: 265282:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [ 8761.816485] LustreError: 267247:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787369252 with bad export cookie 12737360739050376791 [ 8761.826688] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8761.830697] LustreError: 267247:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8762.331784] Lustre: server umount lustre-MDT0001 complete [ 8775.185269] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 8778.434240] Key type lgssc unregistered [ 8778.749588] LNet: 269011:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8778.757570] LNetError: 269011:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8778.795420] LNet: Removed LNI 192.168.204.129@tcp [ 8779.713166] Key type .llcrypt unregistered [ 8779.716492] Key type ._llcrypt unregistered [ 8812.040296] Key type ._llcrypt registered [ 8812.046262] Key type .llcrypt registered [ 8812.181318] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 8813.481020] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8813.689601] alg: No test for adler32 (adler32-zlib) [ 8814.934775] Lustre: Lustre: Build Version: 2.17.57_45_g4630995 [ 8815.319735] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 8817.056288] Key type lgssc registered [ 8818.725641] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8826.635514] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8826.647268] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8828.080883] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8833.658903] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 8837.615161] LustreError: 270476:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8837.646555] LustreError: 270476:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8842.737466] LustreError: 270477:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8844.367856] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8844.891746] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8850.902248] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 8860.393663] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8866.840803] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8875.439762] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8875.855055] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8883.225647] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 8891.545680] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8892.773816] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8898.022720] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8898.025438] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8903.142167] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8903.155972] Lustre: Skipped 1 previous similar message [ 8908.261941] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8908.275399] Lustre: Skipped 1 previous similar message [ 8909.281399] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8909.482401] Lustre: server umount lustre-OST0000 complete [ 8916.449950] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8916.458696] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8916.486883] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8916.497504] Lustre: Skipped 1 previous similar message [ 8918.500589] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8923.620474] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8923.633172] Lustre: Skipped 3 previous similar messages [ 8928.237050] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8928.589150] Lustre: server umount lustre-MDT0000 complete [ 8928.739130] LustreError: 270476:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8928.755464] LustreError: 270476:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8933.004097] LustreError: 270458:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787369423 with bad export cookie 12121726743808665147 [ 8933.025643] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8933.580598] Lustre: server umount lustre-MDT0001 complete [ 8946.533985] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 8950.027311] Key type lgssc unregistered [ 8950.466333] LNet: 274068:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8950.476310] LNetError: 274068:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8950.507764] LNet: Removed LNI 192.168.204.129@tcp [ 8951.746542] Key type .llcrypt unregistered [ 8951.748836] Key type ._llcrypt unregistered [ 8971.475832] Key type ._llcrypt registered [ 8971.479937] Key type .llcrypt registered [ 8971.578175] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 8972.642442] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8972.675230] alg: No test for adler32 (adler32-zlib) [ 8973.796418] Lustre: Lustre: Build Version: 2.17.57_45_g4630995 [ 8974.103239] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 8975.824246] Key type lgssc registered [ 8977.311778] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8988.363312] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8988.400925] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9004.066415] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 9009.189159] LustreError: 275262:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.129@tcp: failed processing log, type 4: rc = -110 [ 9035.938652] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9043.355880] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 9052.009374] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9054.625354] Lustre: server umount lustre-OST0000 complete [ 9076.437430] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 9079.977757] Key type lgssc unregistered [ 9080.216216] LNet: 276898:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9080.225482] LNetError: 276898:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9080.248751] LNet: Removed LNI 192.168.204.129@tcp [ 9081.175098] Key type .llcrypt unregistered [ 9081.183256] Key type ._llcrypt unregistered [ 9124.899215] Key type ._llcrypt registered [ 9124.901784] Key type .llcrypt registered [ 9125.126262] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 9126.727298] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9126.755622] alg: No test for adler32 (adler32-zlib) [ 9128.082530] Lustre: Lustre: Build Version: 2.17.57_45_g4630995 [ 9128.414611] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 9130.187099] Key type lgssc registered [ 9131.594250] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9147.437598] loop0: detected capacity change from 0 to 204800000 [ 9147.758687] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9147.781577] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9148.202330] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9149.262980] loop0: detected capacity change from 0 to 204800000 [ 9149.526703] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9149.549092] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9149.975416] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9151.051375] loop0: detected capacity change from 0 to 204800000 [ 9151.264655] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9151.284870] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9151.438555] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9152.731782] loop0: detected capacity change from 0 to 204800000 [ 9153.011707] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9153.027315] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9153.190554] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9154.293560] loop0: detected capacity change from 0 to 204800000 [ 9154.323974] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9159.037159] loop0: detected capacity change from 0 to 204800000 [ 9159.066989] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9164.268238] loop0: detected capacity change from 0 to 204800000 [ 9164.320131] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9169.287486] loop0: detected capacity change from 0 to 204800000 [ 9169.336817] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9174.448393] loop0: detected capacity change from 0 to 204800000 [ 9174.571795] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9174.615047] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9174.724544] Lustre: 280322:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9174.740281] Lustre: 280322:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9174.760445] Lustre: 280322:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9174.781687] Lustre: 280322:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 9174.799309] Lustre: 280322:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 9174.816052] Lustre: 280322:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 9174.838476] Lustre: 280322:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9174.857389] Lustre: 280322:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 9174.887145] Lustre: 280322:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 9174.905414] Lustre: 280322:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 9174.929487] Lustre: 280322:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 9174.956724] Lustre: 280322:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 9174.968480] Lustre: 280322:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 9174.988284] Lustre: 280322:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 9181.596122] Lustre: server umount MGS complete [ 9182.569206] loop0: detected capacity change from 0 to 204800000 [ 9182.665859] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9183.312911] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9183.365736] Lustre: lustre-MDT0000: Aborting client recovery [ 9184.572947] loop1: detected capacity change from 0 to 204800000 [ 9184.715275] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9184.868692] Lustre: 280753:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9184.895871] Lustre: 280753:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 9184.915317] Lustre: 280753:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 9184.940192] Lustre: 280753:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9184.967959] Lustre: 280753:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 9184.992975] Lustre: 280753:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9185.005969] Lustre: 280753:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9185.048743] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9185.253489] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9185.317280] Lustre: lustre-MDT0001: Aborting client recovery [ 9186.201974] loop2: detected capacity change from 0 to 204800000 [ 9186.435650] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9186.561860] Lustre: 280832:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9186.586934] Lustre: 280832:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9186.604943] Lustre: 280832:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9186.638508] Lustre: 280832:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9186.660137] Lustre: 280832:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9186.675848] Lustre: 280832:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9186.705515] Lustre: 280832:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9186.949481] Lustre: Found index 0 for lustre-OST0000, updating log [ 9187.001166] Lustre: 280850:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9187.075042] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9187.107269] Lustre: lustre-OST0000: Aborting recovery [ 9188.224152] loop3: detected capacity change from 0 to 204800000 [ 9188.452049] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9188.614968] Lustre: 280916:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9188.627776] Lustre: 280916:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9188.648270] Lustre: 280916:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9188.670065] Lustre: 280916:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9188.678827] Lustre: 280916:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9188.687272] Lustre: 280916:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9188.705624] Lustre: 280916:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9188.739978] Lustre: Found index 1 for lustre-OST0001, updating log [ 9188.768557] Lustre: 280922:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9188.910394] Lustre: lustre-OST0001: Aborting recovery [ 9193.976894] LustreError: 280697:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 9194.985508] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9194.991269] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9195.096294] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 9195.109851] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 9196.015912] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 9196.030208] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 9197.750415] LustreError: 280664:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1064us: rc = -2 [ 9198.312503] LustreError: 280665:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1980us: rc = -2 [ 9198.325832] LustreError: 280665:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 22 previous similar messages [ 9211.362480] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9211.367427] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9211.383224] Lustre: Skipped 3 previous similar messages [ 9211.384321] Lustre: Skipped 2 previous similar messages [ 9216.483161] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9216.492619] Lustre: Skipped 2 previous similar messages [ 9218.039710] Lustre: server umount lustre-MDT0000 complete [ 9219.237235] LustreError: 280648:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787369710 with bad export cookie 4005942494686191561 [ 9219.241484] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9219.259478] LustreError: 280648:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9221.601220] LustreError: 281155:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9221.603379] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9221.616586] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9221.616595] Lustre: Skipped 2 previous similar messages [ 9221.633335] LustreError: 281155:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9221.668452] Lustre: Skipped 1 previous similar message [ 9224.610848] LustreError: 280662:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9224.617431] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9224.645679] LustreError: 280662:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9224.654600] Lustre: Skipped 2 previous similar messages [ 9226.142921] Lustre: server umount lustre-MDT0001 complete [ 9233.530405] Lustre: server umount lustre-OST0000 complete [ 9249.248532] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9249.558274] Lustre: server umount lustre-OST0001 complete [ 9258.703742] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 23:35:47 (1787369747) [ 9292.245364] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [ 9305.629176] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 9343.897500] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9344.211682] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9344.241771] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9344.369645] Lustre: lustre-MDT0000: new disk, initializing [ 9344.520120] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9344.532178] Lustre: Skipped 1 previous similar message [ 9344.571843] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9350.567698] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 9363.254730] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9363.437868] Lustre: 284995:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9363.511196] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9363.516604] Lustre: Skipped 1 previous similar message [ 9363.642964] Lustre: lustre-MDT0001: new disk, initializing [ 9363.807589] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9363.852149] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9363.864037] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9368.921762] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 9380.784862] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9388.723703] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9397.924252] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9398.313261] Lustre: lustre-OST0000: new disk, initializing [ 9398.317267] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9398.322443] Lustre: 286970:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9398.390523] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9403.986819] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9404.003791] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9404.126475] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9407.005715] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 9419.415655] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9425.694264] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9436.346673] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9440.363210] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9501.666445] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9501.678880] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9501.694639] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9501.710850] Lustre: Skipped 1 previous similar message [ 9501.722906] Lustre: Skipped 1 previous similar message [ 9504.264547] Lustre: server umount lustre-OST0000 complete [ 9511.905669] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9511.922468] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9511.935141] Lustre: Skipped 1 previous similar message [ 9511.958359] Lustre: Skipped 1 previous similar message [ 9515.274699] Lustre: server umount lustre-MDT0000 complete [ 9517.028499] LustreError: 285008:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9517.046557] LustreError: 285008:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9520.112626] LustreError: 284988:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787370011 with bad export cookie 4005942494686194270 [ 9520.127992] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9520.129055] LustreError: 284988:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9520.576855] Lustre: server umount lustre-MDT0001 complete [ 9532.345771] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9537.916796] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 9543.987175] Lustre: server umount MGS complete [ 9559.185626] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9559.803261] LustreError: 290740:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9559.846974] LustreError: 290753:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9559.907240] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9565.165570] LustreError: 290741:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9565.482865] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 9574.376754] LustreError: 290740:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9574.418972] LustreError: 290740:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9576.589259] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9583.167090] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 9593.473365] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9601.786770] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9609.833721] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9610.314956] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9610.334692] Lustre: Skipped 1 previous similar message [ 9618.306584] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 9628.058406] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9665.510643] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9665.520252] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9665.530771] LustreError: Skipped 1 previous similar message [ 9665.533982] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9665.549826] Lustre: Skipped 1 previous similar message [ 9665.585062] Lustre: Skipped 1 previous similar message [ 9670.606954] Lustre: server umount lustre-OST0000 complete [ 9675.748407] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9681.869104] Lustre: server umount lustre-MDT0000 complete [ 9685.986212] LustreError: 291685:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9686.022453] LustreError: 291685:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9687.636091] LustreError: 290722:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787370178 with bad export cookie 4005942494686196111 [ 9687.648488] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9687.654632] LustreError: 290722:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9688.033024] Lustre: server umount lustre-MDT0001 complete [ 9701.384921] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 9704.621660] Key type lgssc unregistered [ 9705.025801] LNet: 294693:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9705.042374] LNetError: 294693:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9705.076065] LNet: Removed LNI 192.168.204.129@tcp [ 9706.068607] Key type .llcrypt unregistered [ 9706.072151] Key type ._llcrypt unregistered [ 9753.547809] Key type ._llcrypt registered [ 9753.558637] Key type .llcrypt registered [ 9753.665399] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [ 9772.399837] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 9773.808400] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9773.896706] alg: No test for adler32 (adler32-zlib) [ 9775.470473] Lustre: Lustre: Build Version: 2.17.57_45_g4630995 [ 9775.788434] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 9777.560233] Key type lgssc registered [ 9778.839812] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9823.824467] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9823.844329] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9825.061267] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9825.088907] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9825.169381] Lustre: lustre-MDT0000: new disk, initializing [ 9825.299301] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9825.317310] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9830.878581] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 9843.220625] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9843.293121] Lustre: 298656:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9843.344057] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9843.347649] Lustre: Skipped 1 previous similar message [ 9843.411655] Lustre: lustre-MDT0001: new disk, initializing [ 9843.496759] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9843.537344] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9843.557838] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9848.195730] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 9860.316552] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9867.772773] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9875.804626] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9876.301743] Lustre: lustre-OST0000: new disk, initializing [ 9876.306417] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9876.323289] Lustre: 300627:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9876.475533] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9883.328316] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9883.359505] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9883.564695] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9885.280771] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 9899.818386] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9908.482548] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9921.863174] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9927.785282] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9933.104643] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9980.902451] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9980.904566] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9980.920243] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9980.953013] Lustre: Skipped 1 previous similar message [ 9984.388379] Lustre: server umount lustre-OST0000 complete [ 9991.149264] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9991.168212] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9991.175600] Lustre: Skipped 1 previous similar message [ 9994.816403] Lustre: server umount lustre-MDT0000 complete [ 9998.990728] LustreError: 298651:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787370489 with bad export cookie 664766721554938595 [ 9998.997492] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9999.006376] LustreError: 298651:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9999.030411] LustreError: 296879:0:(import.c:717:ptlrpc_connect_import_locked()) already connecting [ 9999.550955] Lustre: server umount lustre-MDT0001 complete [10010.233035] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10016.509191] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [10023.257938] Lustre: server umount MGS complete [10038.034900] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10038.527466] LustreError: 304455:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10038.545675] LustreError: 304455:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10038.567201] LustreError: 304468:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10038.636569] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10043.828810] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [10043.882522] LustreError: 304456:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10048.996984] LustreError: 304455:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10053.936044] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10054.113969] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [10054.318144] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10059.589637] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [10068.108762] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10074.832041] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10082.331245] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10082.643323] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10089.677143] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [10099.069134] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10130.915370] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10130.918559] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10130.924935] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10130.931913] Lustre: Skipped 2 previous similar messages [10136.039654] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10136.047257] Lustre: Skipped 2 previous similar messages [10136.740392] Lustre: server umount lustre-OST0000 complete [10141.157543] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10141.180546] Lustre: Skipped 1 previous similar message [10146.274676] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10146.285328] Lustre: Skipped 3 previous similar messages [10147.145630] Lustre: server umount lustre-MDT0000 complete [10150.985385] LustreError: 304435:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787370641 with bad export cookie 664766721554940513 [10151.002431] LustreError: 304435:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10151.005217] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10151.383697] Lustre: server umount lustre-MDT0001 complete [10163.016730] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [10166.240608] Key type lgssc unregistered [10166.640950] LNet: 308403:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10166.651363] LNetError: 308403:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10166.685426] LNet: Removed LNI 192.168.204.129@tcp [10167.666535] Key type .llcrypt unregistered [10167.671174] Key type ._llcrypt unregistered [10191.063515] Key type ._llcrypt registered [10191.066073] Key type .llcrypt registered [10191.250905] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [10205.632437] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [10206.616680] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10206.770844] alg: No test for adler32 (adler32-zlib) [10207.895654] Lustre: Lustre: Build Version: 2.17.57_45_g4630995 [10208.225512] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [10209.936193] Key type lgssc registered [10211.587461] Lustre: Echo OBD driver; http://www.lustre.org/ [10239.009498] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [10256.984803] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10257.042851] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10258.389662] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10258.451879] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10258.600957] Lustre: lustre-MDT0000: new disk, initializing [10258.727508] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10258.757747] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10264.611654] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [10278.800524] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10279.087441] Lustre: lustre-OST0000: new disk, initializing [10279.090851] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10279.094527] Lustre: Skipped 1 previous similar message [10279.102566] Lustre: 312455:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10279.198356] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10286.256593] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [10288.683335] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [10288.697085] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [10288.736096] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [11307.234669] Lustre: server umount lustre-MDT0000 complete [11311.443398] LustreError: 313563:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787371802 with bad export cookie 5289026736565957946 [11311.457523] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11321.942803] Lustre: server umount lustre-OST0000 complete [11352.643610] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [11364.307557] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [11410.229579] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11410.461878] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11410.484067] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11410.539314] Lustre: lustre-MDT0000: new disk, initializing [11410.604262] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11410.625649] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11416.336477] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [11428.956357] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11429.109699] Lustre: 316839:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11429.184364] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11429.194923] Lustre: Skipped 1 previous similar message [11429.362231] Lustre: lustre-MDT0001: new disk, initializing [11429.478315] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11429.533594] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11429.541612] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11434.872320] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [11445.949364] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11453.554922] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11461.244560] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11461.616089] Lustre: lustre-OST0000: new disk, initializing [11461.620657] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11461.628341] Lustre: 318810:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11461.725118] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11468.672134] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [11470.422941] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11470.433047] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11470.572571] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11480.061709] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11485.609343] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11485.838292] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11491.273872] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11491.562328] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11496.421668] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11496.429323] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11496.449813] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11496.450555] Lustre: Skipped 1 previous similar message [11496.469677] Lustre: Skipped 1 previous similar message [11499.849642] Lustre: server umount lustre-OST0000 complete [11506.147797] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11506.160713] LustreError: Skipped 1 previous similar message [11506.168630] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11506.193983] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11510.757642] Lustre: server umount lustre-MDT0000 complete [11515.268089] LustreError: 317634:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787372006 with bad export cookie 5289026736568898758 [11515.291813] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11515.295238] LustreError: 317634:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11515.715413] Lustre: server umount lustre-MDT0001 complete [11520.036857] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [11522.279840] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 00:13:30 (1787372010) [11530.051801] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11530.600292] LustreError: 321124:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11530.623138] LustreError: 321124:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11530.690221] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11535.842915] LustreError: 321125:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11536.538297] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [11540.963675] LustreError: 321124:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11546.083672] LustreError: 321125:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11546.493147] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11546.993538] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11552.748308] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [11562.520566] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11569.429543] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11577.014602] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11577.422222] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11584.794770] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [11592.784163] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11601.400991] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11601.534836] Lustre: lustre-OST0001: new disk, initializing [11601.537695] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [11601.542553] Lustre: 323959:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11601.626779] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11607.643668] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [11607.652322] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [11607.706026] LustreError: 324258:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11608.584339] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [11608.738894] LustreError: 324258:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11608.758773] LustreError: 324258:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [11609.834346] LustreError: 324258:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11609.850036] LustreError: 324258:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [11611.952178] LustreError: 324258:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11611.965209] LustreError: 324258:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [11616.110108] LustreError: 324258:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11616.118486] LustreError: 324258:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [11620.056967] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11620.223899] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11624.420751] LustreError: 324264:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11624.431245] LustreError: 324264:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 16 previous similar messages [11626.821861] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff91e546781000.ost_server_uuid 50 [11628.656572] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff91e546781000.ost_server_uuid in FULL state after 0 sec [11635.402679] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff91e546781000.ost_server_uuid 50 [11637.482640] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff91e546781000.ost_server_uuid in IDLE state after 0 sec [11639.887255] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [11655.138580] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11655.140741] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11655.159593] LustreError: Skipped 1 previous similar message [11655.168599] Lustre: Skipped 1 previous similar message [11655.169955] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11655.182619] Lustre: Skipped 2 previous similar messages [11657.204445] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11657.220737] Lustre: Skipped 2 previous similar messages [11659.292407] Lustre: server umount lustre-OST0001 complete [11660.614398] LustreError: 322944:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.204.29@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11667.438582] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11667.451507] Lustre: Skipped 1 previous similar message [11667.454963] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11667.462365] Lustre: Skipped 1 previous similar message [11672.467229] Lustre: server umount lustre-OST0000 complete [11677.668351] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11677.672116] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11677.684363] Lustre: Skipped 2 previous similar messages [11677.694111] Lustre: Skipped 1 previous similar message [11683.215772] Lustre: server umount lustre-MDT0000 complete [11687.849397] LustreError: 323958:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787372178 with bad export cookie 5289026736568899640 [11687.858026] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11687.870966] LustreError: 323958:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11688.216229] Lustre: server umount lustre-MDT0001 complete [11701.052952] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [11704.121330] Key type lgssc unregistered [11704.500462] LNet: 326654:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11704.513227] LNetError: 326654:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11704.538371] LNet: Removed LNI 192.168.204.129@tcp [11705.475836] Key type .llcrypt unregistered [11705.481601] Key type ._llcrypt unregistered [11737.521193] Key type ._llcrypt registered [11737.527765] Key type .llcrypt registered [11737.765993] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [11739.009585] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11739.036412] alg: No test for adler32 (adler32-zlib) [11740.113236] Lustre: Lustre: Build Version: 2.17.57_45_g4630995 [11740.273527] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [11741.904706] Key type lgssc registered [11743.365852] Lustre: Echo OBD driver; http://www.lustre.org/ [11751.807122] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11751.830605] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11753.363484] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11758.556177] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [11763.700271] LustreError: 328078:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11763.723971] LustreError: 328078:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11766.813292] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11768.803365] LustreError: 328097:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11773.930632] LustreError: 328078:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11774.791719] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11775.238193] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11780.471666] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [11791.255753] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11791.619593] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11798.764610] LustreError: 329759:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11798.788051] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [11801.073993] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [11806.185462] LustreError: 329760:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11806.211376] LustreError: 329760:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11810.795973] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11816.435044] LustreError: 330023:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11816.474512] LustreError: 330023:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11818.435994] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11818.703294] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11824.117649] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [11825.458310] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [11833.577080] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11841.457589] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [11841.712638] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11957.233709] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11957.237729] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11957.254851] LustreError: Skipped 1 previous similar message [11957.263130] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11957.284094] Lustre: Skipped 1 previous similar message [11957.297361] Lustre: Skipped 1 previous similar message [11958.683055] Lustre: server umount lustre-OST0001 complete [11962.339212] LustreError: 331048:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11962.376854] LustreError: 331048:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11964.907569] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11964.928444] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11964.962190] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11967.473912] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11967.476059] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11971.075839] Lustre: server umount lustre-OST0000 complete [11977.704124] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11977.722092] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11977.729247] Lustre: Skipped 1 previous similar message [11977.755237] Lustre: Skipped 2 previous similar messages [11981.655965] Lustre: server umount lustre-MDT0000 complete [11986.045729] LustreError: 328060:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787372476 with bad export cookie 6359488194273897786 [11986.050235] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11986.055996] LustreError: 328060:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11986.639249] Lustre: server umount lustre-MDT0001 complete [11999.231072] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [12001.861596] Key type lgssc unregistered [12002.225289] LNet: 333027:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12002.238697] LNetError: 333027:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12002.257971] LNet: Removed LNI 192.168.204.129@tcp [12003.178909] Key type .llcrypt unregistered [12003.182491] Key type ._llcrypt unregistered [12050.001621] Key type ._llcrypt registered [12050.004925] Key type .llcrypt registered [12050.099534] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [12051.099673] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12051.135907] alg: No test for adler32 (adler32-zlib) [12052.156975] Lustre: Lustre: Build Version: 2.17.57_45_g4630995 [12052.357830] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [12054.032163] Key type lgssc registered [12055.010150] Lustre: Echo OBD driver; http://www.lustre.org/ [12062.681655] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12062.696799] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12064.025557] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12067.827195] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [12073.442995] LustreError: 335318:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12073.496828] LustreError: 335318:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12078.563041] LustreError: 335317:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12078.827762] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12079.316724] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12085.102513] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [12089.390925] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [12097.795119] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12098.499962] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12104.625439] LustreError: 336809:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12105.007593] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [12108.709499] LustreError: 336810:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12108.723565] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [12108.727043] LustreError: 336810:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12113.900774] LustreError: 337131:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12113.928164] LustreError: 337131:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12114.007189] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12114.191571] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12119.552097] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [12120.387864] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [12130.044494] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12135.630941] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [12150.243969] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12150.249365] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12150.274984] Lustre: Skipped 2 previous similar messages [12150.286920] Lustre: Skipped 3 previous similar messages [12155.366692] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12155.373665] Lustre: Skipped 3 previous similar messages [12155.742236] Lustre: server umount lustre-MDT0000 complete [12159.391981] LustreError: 335298:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787372650 with bad export cookie 13843263470013803967 [12159.400035] LustreError: 335298:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [12159.400602] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12159.773490] Lustre: server umount lustre-MDT0001 complete [12174.503821] Lustre: server umount lustre-OST0000 complete [12176.864127] Lustre: 334583:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787372651/real 1787372651] req@ffff96781ed57100 x1874195951713792/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787372667 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [12176.875631] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12176.881059] Lustre: Skipped 1 previous similar message [12177.942563] Lustre: server umount lustre-OST0001 complete [12198.129698] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [12210.485838] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12211.018911] LustreError: 340585:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12211.179597] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12215.891289] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [12225.977273] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12226.612524] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12231.196822] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [12234.106692] Lustre: 341726:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [12241.236711] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12248.136451] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [12251.814779] LustreError: 342080:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12251.853369] LustreError: 342080:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12255.947948] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [12258.262558] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12258.543134] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12258.553525] Lustre: Skipped 1 previous similar message [12263.919326] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [12265.411507] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [12273.561166] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12278.152513] Lustre: 343599:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [12282.849925] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12282.856757] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12282.871729] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12289.148954] Lustre: server umount lustre-MDT0000 complete [12289.508967] LustreError: 343183:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12289.522771] LustreError: 343183:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [12293.158204] LustreError: 341728:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787372784 with bad export cookie 13843263470013805661 [12293.165400] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12293.172409] LustreError: 341728:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12293.612162] Lustre: server umount lustre-MDT0001 complete [12307.457915] Lustre: server umount lustre-OST0000 complete [12310.501518] Lustre: 334582:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787372785/real 1787372785] req@ffff9677140ed880 x1874195951763968/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787372801 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [12310.527693] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12310.537735] Lustre: Skipped 3 previous similar messages [12311.720839] Lustre: server umount lustre-OST0001 complete [12315.744001] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [12317.649604] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [12319.777329] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 00:26:48 (1787372808) [12349.548618] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [12360.377511] loop0: detected capacity change from 0 to 1099511627776 [12361.237036] SGI XFS with ACLs, security attributes, quota, no debug enabled [12361.247438] *** VALIDATE xfs *** [12361.257980] XFS (loop0): Mounting V5 Filesystem [12361.315628] XFS (loop0): Ending clean mount [12362.743090] loop1: detected capacity change from 0 to 18691697672192 [12486.182394] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [12497.869947] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [12537.079798] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12537.455368] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12537.488694] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12537.626222] Lustre: lustre-MDT0000: new disk, initializing [12537.743530] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12537.788515] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12542.939944] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [12556.116326] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12556.316848] Lustre: 348737:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12556.382979] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12556.393092] Lustre: Skipped 1 previous similar message [12556.508344] Lustre: lustre-MDT0001: new disk, initializing [12556.653941] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12556.671055] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12562.188137] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [12572.985101] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12580.143530] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12587.065885] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12587.384898] Lustre: lustre-OST0000: new disk, initializing [12587.392425] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12587.398497] Lustre: 350706:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12589.261716] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12589.279179] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12589.408879] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12594.710439] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [12605.132398] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12610.555189] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12610.865262] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12616.439377] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [12616.631762] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12620.260738] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12620.263726] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12620.267151] LustreError: Skipped 1 previous similar message [12620.269775] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12620.274832] Lustre: Skipped 1 previous similar message [12620.277486] Lustre: Skipped 4 previous similar messages [12625.400306] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12625.414644] Lustre: Skipped 2 previous similar messages [12633.057619] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12633.556791] Lustre: server umount lustre-OST0000 complete [12640.740931] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12640.783628] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12640.797713] Lustre: Skipped 3 previous similar messages [12645.523332] Lustre: server umount lustre-MDT0000 complete [12645.861066] LustreError: 348744:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12645.872964] LustreError: 348744:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12649.821402] LustreError: 348730:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787373140 with bad export cookie 13843263470013807222 [12649.840889] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12649.843669] LustreError: 348730:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [12650.327297] Lustre: server umount lustre-MDT0001 complete [12658.492985] XFS (loop0): Unmounting Filesystem [12663.891213] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 00:32:32 (1787373152) [12671.813968] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12672.568072] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12672.577840] Lustre: Skipped 2 previous similar messages [12677.835595] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [12686.754660] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12691.859212] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [12699.898375] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12706.570453] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12712.568359] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12718.910903] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [12726.766472] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12741.611649] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12741.627370] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12741.627918] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12741.628329] Lustre: Skipped 2 previous similar messages [12746.754333] Lustre: server umount lustre-OST0000 complete [12757.594586] Lustre: server umount lustre-MDT0000 complete [12762.947251] LustreError: 353252:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787373253 with bad export cookie 13843263470013808104 [12762.965751] LustreError: 353252:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [12762.969341] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12763.297296] Lustre: server umount lustre-MDT0001 complete [12775.922110] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [12778.702667] Key type lgssc unregistered [12779.046215] LNet: 356902:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12779.052830] LNetError: 356902:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12779.070223] LNet: Removed LNI 192.168.204.129@tcp [12779.928197] Key type .llcrypt unregistered [12779.930621] Key type ._llcrypt unregistered [12809.831336] Key type ._llcrypt registered [12809.832945] Key type .llcrypt registered [12809.966640] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [12811.130514] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12811.198520] alg: No test for adler32 (adler32-zlib) [12812.328701] Lustre: Lustre: Build Version: 2.17.57_45_g4630995 [12812.665642] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [12814.376862] Key type lgssc registered [12815.651798] Lustre: Echo OBD driver; http://www.lustre.org/ [12824.075575] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12824.099517] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12825.579944] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12830.788310] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [12835.816979] LustreError: 358470:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12835.850435] LustreError: 358470:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12840.200488] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12840.596024] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12845.239306] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [12853.683407] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12860.636746] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12868.560752] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12868.994226] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12876.438557] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [12885.450223] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12895.719226] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12895.738028] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12895.743425] Lustre: Skipped 1 previous similar message [12898.158779] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.29@tcp (stopping) [12900.126461] Lustre: server umount lustre-MDT0001 complete [12900.836956] LustreError: 358470:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12900.858900] LustreError: 358470:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12903.231163] LustreError: 358466:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.29@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12905.962944] LustreError: 358471:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12906.002954] LustreError: 358471:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12911.074771] LustreError: 359199:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12911.104684] LustreError: 359199:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12911.191270] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12911.263361] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12911.275797] Lustre: Found index 1 for lustre-MDT0001, updating log [12911.342876] Lustre: 358459:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12911.616775] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12916.511461] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [12916.715260] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12917.249164] LustreError: 361951:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12917.257428] LustreError: 361951:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12917.269945] LustreError: 358462:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12917.330096] LustreError: 361955:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.204.129@tcp: cfg command failed: rc = -17 [12917.337562] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [12917.337562] [12926.951110] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12926.952658] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12926.962752] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12926.990884] Lustre: Skipped 2 previous similar messages [12932.067761] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12932.072941] Lustre: Skipped 3 previous similar messages [12932.840874] Lustre: server umount lustre-MDT0001 complete [12933.951306] LustreError: 358467:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.29@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12944.249467] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12944.403747] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12944.411329] Lustre: Found index 1 for lustre-MDT0001, updating log [12944.457625] Lustre: 358460:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12944.932826] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12949.864751] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [12950.003194] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12953.092655] LustreError: 363141:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12953.110775] LustreError: 363141:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12953.119350] LustreError: 358462:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12953.123798] LustreError: 358462:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12953.211688] LustreError: 363150:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12953.217428] LustreError: 363150:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12953.221968] LustreError: 363150:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.204.129@tcp: cfg command failed: rc = -17 [12953.228544] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.204.129@tcp [12953.228544] [13256.556114] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.29@tcp (stopping) [13257.187507] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13257.201092] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13257.219713] Lustre: Skipped 1 previous similar message [13262.575536] Lustre: server umount lustre-MDT0001 complete [13266.757681] LustreError: 358465:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.29@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13266.790722] LustreError: 358465:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [13275.442699] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13275.587726] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13275.600058] Lustre: Found index 1 for lustre-MDT0001, updating log [13275.644560] Lustre: 358457:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13276.048608] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13281.209078] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [13281.256204] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13283.848952] LustreError: 371636:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13283.857932] LustreError: 371636:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13283.871636] LustreError: 358462:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13283.884395] LustreError: 358462:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13283.962432] LustreError: 371643:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [13283.980153] LustreError: 371643:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.204.129@tcp: cfg command failed: rc = -114 [13283.990759] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [13283.990759] [13603.811856] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13603.811893] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13603.822229] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13603.826937] Lustre: Skipped 2 previous similar messages [13603.841836] Lustre: Skipped 6 previous similar messages [13608.820093] Lustre: server umount lustre-MDT0000 complete [13608.933970] LustreError: 379043:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13608.962027] LustreError: 379043:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [13612.789937] LustreError: 358452:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787374103 with bad export cookie 10940810243804264123 [13612.795553] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13612.807946] LustreError: 358452:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [13614.050982] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13622.188355] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13622.196894] Lustre: Skipped 3 previous similar messages [13627.360578] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [13627.734461] Lustre: server umount lustre-MDT0001 complete [13632.637059] Lustre: server umount lustre-OST0000 complete [13643.792075] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [13654.037792] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [13697.340925] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 00:49:45 (1787374185) [13705.428674] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13705.632275] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13705.686954] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13705.788426] Lustre: lustre-MDT0000: new disk, initializing [13705.887274] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13705.902709] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13710.732550] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [13722.194483] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13722.337321] Lustre: 382106:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13722.393981] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13722.398603] Lustre: Skipped 1 previous similar message [13722.468782] Lustre: lustre-MDT0001: new disk, initializing [13722.581946] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13722.616736] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13722.627251] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13727.939329] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [13737.876299] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13744.373764] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13752.054582] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13752.553634] Lustre: lustre-OST0000: new disk, initializing [13752.560827] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13752.574636] Lustre: 384076:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13752.781280] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13760.633301] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [13762.650177] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13762.655573] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13762.831848] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13772.146513] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13780.970863] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [13780.992304] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13781.027349] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13781.034369] Lustre: Skipped 1 previous similar message [13781.541765] Lustre: server umount lustre-MDT0000 complete [13783.528927] LustreError: 383185:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13783.563548] LustreError: 383185:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [13785.127876] LustreError: 382912:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787374276 with bad export cookie 10940810243804266076 [13785.139465] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13785.140706] LustreError: 382912:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [13785.163376] LustreError: 382915:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [13785.413807] Lustre: server umount lustre-MDT0001 complete [13799.640428] Lustre: server umount lustre-OST0000 complete [13832.880959] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [13842.951549] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [13882.050963] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 13591 sec ========= 00:52:50 (1787374370) [13883.806895] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 00:52:52 (1787374372) === [13888.444213] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 00:52:56 (1787374376) === [13942.036600] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [13945.493388] Key type lgssc unregistered [13945.767553] LNet: 390449:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13945.777874] LNetError: 390449:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13945.803979] LNet: Removed LNI 192.168.204.129@tcp [13946.999171] Key type .llcrypt unregistered [13947.001584] Key type ._llcrypt unregistered