[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 1169572779 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.002000] x2apic enabled [ 0.002000] Switched APIC routing to physical x2apic. [ 0.002000] kvm-guest: setup PV IPIs [ 0.002000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.002000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.002019] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.003014] pid_max: default: 32768 minimum: 301 [ 0.004058] LSM: Security Framework initializing [ 0.005058] Yama: becoming mindful. [ 0.007033] SELinux: Initializing. [ 0.008059] *** VALIDATE selinux *** [ 0.017943] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.022248] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.024074] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.025105] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.026107] *** VALIDATE tmpfs *** [ 0.027446] *** VALIDATE proc *** [ 0.029241] *** VALIDATE cgroup *** [ 0.030009] *** VALIDATE cgroup2 *** [ 0.031236] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.032133] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.033007] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.034026] Spectre V2 : User space: Vulnerable [ 0.035007] Speculative Store Bypass: Vulnerable [ 0.038283] debug: unmapping init [mem 0xffffffffaae59000-0xffffffffaae60fff] [ 0.040904] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.042347] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.043019] ... version: 2 [ 0.044010] ... bit width: 48 [ 0.045009] ... generic registers: 4 [ 0.046008] ... value mask: 0000ffffffffffff [ 0.047010] ... max period: 00007fffffffffff [ 0.048008] ... fixed-purpose events: 3 [ 0.049008] ... event mask: 000000070000000f [ 0.050249] rcu: Hierarchical SRCU implementation. [ 0.052339] smp: Bringing up secondary CPUs ... [ 0.053495] x86: Booting SMP configuration: [ 0.054017] .... node #0, CPUs: #1 #2 #3 [ 0.062164] smp: Brought up 1 node, 4 CPUs [ 0.064013] smpboot: Max logical packages: 1 [ 0.065010] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.095524] node 0 deferred pages initialised in 29ms [ 0.099249] devtmpfs: initialized [ 0.100280] x86/mm: Memory block size: 128MB [ 0.102816] gcov: version magic: 0x41383552 [ 0.104008] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.105067] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.106465] pinctrl core: initialized pinctrl subsystem [ 0.107220] [ 0.107788] ************************************************************* [ 0.108013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.109012] ** ** [ 0.110013] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.111012] ** ** [ 0.112011] ** This means that this kernel is built to expose internal ** [ 0.113024] ** IOMMU data structures, which may compromise security on ** [ 0.114015] ** your system. ** [ 0.115011] ** ** [ 0.116014] ** If you see this message and you are not debugging the ** [ 0.117014] ** kernel, report this immediately to your vendor! ** [ 0.118013] ** ** [ 0.119123] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.120010] ************************************************************* [ 0.121684] NET: Registered protocol family 16 [ 0.122411] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.123144] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.124058] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.125584] cpuidle: using governor menu [ 0.126825] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.127606] PCI: Using configuration type 1 for base access [ 0.128262] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.135249] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.136000] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.138047] cryptd: max_cpu_qlen set to 1000 [ 0.139173] ACPI: Added _OSI(Module Device) [ 0.140000] ACPI: Added _OSI(Processor Device) [ 0.140000] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.140021] ACPI: Added _OSI(Processor Aggregator Device) [ 0.145461] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.152696] ACPI: Interpreter enabled [ 0.154019] ACPI: PM: (supports S0 S3 S4 S5) [ 0.156010] ACPI: Using IOAPIC for interrupt routing [ 0.158108] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.162000] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.173563] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.176034] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.178015] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.182066] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.187000] acpiphp: Slot [2] registered [ 0.189161] acpiphp: Slot [5] registered [ 0.191150] acpiphp: Slot [6] registered [ 0.193170] acpiphp: Slot [7] registered [ 0.194108] acpiphp: Slot [8] registered [ 0.195100] acpiphp: Slot [9] registered [ 0.196105] acpiphp: Slot [10] registered [ 0.197113] acpiphp: Slot [3] registered [ 0.199090] acpiphp: Slot [4] registered [ 0.201089] acpiphp: Slot [11] registered [ 0.202080] acpiphp: Slot [12] registered [ 0.204130] acpiphp: Slot [13] registered [ 0.205088] acpiphp: Slot [14] registered [ 0.207080] acpiphp: Slot [15] registered [ 0.208081] acpiphp: Slot [16] registered [ 0.210080] acpiphp: Slot [17] registered [ 0.211076] acpiphp: Slot [18] registered [ 0.213103] acpiphp: Slot [19] registered [ 0.214099] acpiphp: Slot [20] registered [ 0.216090] acpiphp: Slot [21] registered [ 0.218081] acpiphp: Slot [22] registered [ 0.219071] acpiphp: Slot [23] registered [ 0.220080] acpiphp: Slot [24] registered [ 0.222120] acpiphp: Slot [25] registered [ 0.224092] acpiphp: Slot [26] registered [ 0.225118] acpiphp: Slot [27] registered [ 0.227080] acpiphp: Slot [28] registered [ 0.228087] acpiphp: Slot [29] registered [ 0.230087] acpiphp: Slot [30] registered [ 0.232079] acpiphp: Slot [31] registered [ 0.233046] PCI host bridge to bus 0000:00 [ 0.234015] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.237118] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.240019] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.242018] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.245019] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.247020] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.249178] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.251968] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.255476] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.265015] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.270590] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.274018] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.276016] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.281072] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.284486] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.304363] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.309012] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.313635] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.411013] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.423017] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.429025] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.454067] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.479025] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.496023] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.546024] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.560921] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.567013] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.577017] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.597017] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.628170] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.935022] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.957019] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 1.070023] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 1.097169] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 1.116021] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 1.133022] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 1.168019] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 1.192161] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 1.206020] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 1.222020] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 1.246021] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 1.261000] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 1.270022] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 1.279019] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 1.297023] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 1.309179] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 1.311413] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 1.314456] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 1.317325] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 1.318140] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 1.323138] iommu: Default domain type: Passthrough [ 1.325526] SCSI subsystem initialized [ 1.327145] ACPI: bus type USB registered [ 1.328161] usbcore: registered new interface driver usbfs [ 1.332107] usbcore: registered new interface driver hub [ 1.334114] usbcore: registered new device driver usb [ 1.336206] pps_core: LinuxPPS API ver. 1 registered [ 1.338014] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 1.342047] PTP clock support registered [ 1.345111] EDAC MC: Ver: 3.0.0 [ 1.347200] PCI: Using ACPI for IRQ routing [ 1.348811] NetLabel: Initializing [ 1.349009] NetLabel: domain hash size = 128 [ 1.350018] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 1.351185] NetLabel: unlabeled traffic allowed by default [ 1.352190] vgaarb: loaded [ 1.354288] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 1.355010] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 1.360783] clocksource: Switched to clocksource kvm-clock [ 1.488343] VFS: Disk quotas dquot_6.6.0 [ 1.489805] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.492429] *** VALIDATE ramfs *** [ 1.493799] *** VALIDATE hugetlbfs *** [ 1.495448] pnp: PnP ACPI init [ 1.497679] pnp: PnP ACPI: found 6 devices [ 1.513622] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.516918] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.519188] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.521341] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.524217] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.526959] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.529994] NET: Registered protocol family 2 [ 1.532688] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.538410] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.542187] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.547451] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.550768] TCP: Hash tables configured (established 65536 bind 65536) [ 1.553605] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.556917] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.560161] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.563467] NET: Registered protocol family 1 [ 1.566795] RPC: Registered named UNIX socket transport module. [ 1.569355] RPC: Registered udp transport module. [ 1.571175] RPC: Registered tcp transport module. [ 1.573406] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.576309] NET: Registered protocol family 44 [ 1.577943] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.580362] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.582613] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.585044] PCI: CLS 0 bytes, default 64 [ 1.586850] Unpacking initramfs... [ 3.082184] debug: unmapping init [mem 0xffff88d1fcc54000-0xffff88d1fffbffff] [ 3.086165] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 3.088545] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 3.091466] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.597986] Initialise system trusted keyrings [ 3.599765] Key type blacklist registered [ 3.601603] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.610757] zbud: loaded [ 3.613622] *** VALIDATE nfs *** [ 3.614874] *** VALIDATE nfs4 *** [ 3.616396] pstore: using deflate compression [ 3.619544] Platform Keyring initialized [ 3.752863] NET: Registered protocol family 38 [ 3.755315] Key type asymmetric registered [ 3.757322] Asymmetric key parser 'x509' registered [ 3.759678] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.764021] io scheduler mq-deadline registered [ 3.766022] io scheduler kyber registered [ 3.767849] io scheduler bfq registered [ 3.769972] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.774549] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.778324] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.781337] ACPI: Power Button [PWRF] [ 3.786963] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.794215] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.812418] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.820790] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.835618] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.869060] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.905863] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.911348] Non-volatile memory driver v1.3 [ 3.913066] Linux agpgart interface v0.103 [ 3.943055] virtio_blk virtio1: [vda] 139528 512-byte logical blocks (71.4 MB/68.1 MiB) [ 3.946180] vda: detected capacity change from 0 to 71438336 [ 3.964831] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.968424] vdb: detected capacity change from 0 to 1073741824 [ 3.984396] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.987398] vdc: detected capacity change from 0 to 2621440000 [ 4.001841] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 4.005606] vdd: detected capacity change from 0 to 2621440000 [ 4.023242] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 4.026863] vde: detected capacity change from 0 to 4294967296 [ 4.044748] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 4.048649] vdf: detected capacity change from 0 to 4294967296 [ 4.055797] libphy: Fixed MDIO Bus: probed [ 4.063953] usbcore: registered new interface driver usbserial_generic [ 4.066124] usbserial: USB Serial support registered for generic [ 4.068252] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 4.072719] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 4.074477] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 4.076711] mousedev: PS/2 mouse device common for all mice [ 4.079847] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 4.081310] rtc_cmos 00:05: RTC can wake from S4 [ 4.087268] rtc_cmos 00:05: registered as rtc0 [ 4.088978] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 4.093139] intel_pstate: CPU model not supported [ 4.097912] hid: raw HID events driver (C) Jiri Kosina [ 4.100417] usbcore: registered new interface driver usbhid [ 4.102907] usbhid: USB HID core driver [ 4.105050] drop_monitor: Initializing network drop monitor service [ 4.107027] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 4.109815] Initializing XFRM netlink socket [ 4.111671] NET: Registered protocol family 10 [ 4.119792] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 4.125218] Segment Routing with IPv6 [ 4.128717] NET: Registered protocol family 17 [ 4.130808] mpls_gso: MPLS GSO support [ 4.135874] RAS: Correctable Errors collector initialized. [ 4.138212] AVX version of gcm_enc/dec engaged. [ 4.139617] AES CTR mode by8 optimization enabled [ 4.237938] sched_clock: Marking stable (4237808128, 0)->(5353005864, -1115197736) [ 4.241490] registered taskstats version 1 [ 4.243620] Loading compiled-in X.509 certificates [ 4.245892] zswap: loaded using pool lzo/zbud [ 4.289433] Key type big_key registered [ 4.307477] Key type encrypted registered [ 4.363069] ima: No TPM chip found, activating TPM-bypass! [ 4.366051] ima: Allocated hash algorithm: sha1 [ 4.367675] ima: No architecture policies found [ 4.371954] evm: Initialising EVM extended attributes: [ 4.373515] evm: security.selinux [ 4.374290] evm: security.ima [ 4.374957] evm: security.capability [ 4.375941] evm: HMAC attrs: 0x1 [ 4.379142] rtc_cmos 00:05: setting system clock to 2026-06-09 17:52:55 UTC (1781027575) [ 4.386213] debug: unmapping init [mem 0xffffffffabe03000-0xffffffffabffffff] [ 4.389210] debug: unmapping init [mem 0xffffffffaab82000-0xffffffffaae58fff] [ 4.402078] Write protecting the kernel read-only data: 28672k [ 4.405793] debug: unmapping init [mem 0xffffffffa9203000-0xffffffffa93fffff] [ 4.409278] debug: unmapping init [mem 0xffffffffa9b14000-0xffffffffa9bfffff] [ 4.464473] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 4.473279] systemd[1]: Detected virtualization kvm. [ 4.475795] systemd[1]: Detected architecture x86-64. [ 4.478252] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 4.502659] systemd[1]: No hostname configured. [ 4.504803] systemd[1]: Set hostname to . [ 4.507336] random: systemd: uninitialized urandom read (16 bytes read) [ 4.509966] systemd[1]: Initializing machine ID from random generator. [ 4.557883] random: ln: uninitialized urandom read (6 bytes read) [ 4.660539] random: systemd: uninitialized urandom read (16 bytes read) [ 4.663725] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ 4.670791] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 4.675527] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on udev Control Socket. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Slices. [ OK ] Reached target Initrd Root Device. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Local File Systems. Starting Journal Service... [ OK ] Reached target Timers. [ OK ] Started Memstrack Anylazing Service. Starting Apply Kernel Variables... Starting Setup Virtual Console... Starting Create Volatile Files and Directories... [ OK ] Reached target Sockets. [ OK ] Reached target Paths. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 5.637365] device-mapper: uevent: version 1.0.3 [ 5.639690] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 6.569932] virtio_net virtio0 ens2: renamed from eth0 [ 6.615565] random: fast init done [ 6.689855] scsi host0: ata_piix [ 6.696061] scsi host1: ata_piix [ 6.697857] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 6.700909] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 10.969452] random: crng init done [ 10.970933] random: 7 urandom warning(s) missed due to ratelimiting [ 12.348806] dracut-initqueue[590]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 13.287646] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Timers. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. Stopping udev Kernel Device Manager... [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 15.449691] printk: systemd: 25 output lines suppressed due to ratelimiting [ 16.052709] SELinux: Disabled at runtime. [ 16.386800] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 16.395204] systemd[1]: Detected virtualization kvm. [ 16.396806] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 17.349251] systemd[1]: initrd-switch-root.service: Succeeded. [ 17.352991] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 17.362565] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 17.371494] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 17.376128] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 17.384134] systemd[1]: Starting Journal Service... Starting Journal Service... [ 17.392341] systemd[1]: Created slice system-serial\x2dgetty.slice. [ OK ] Created slice system-serial\x2dgetty.slice. Mounting Huge Pages File System... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target rpc_pipefs.target. [ OK ] Listening on Process Core Dump Socket. [ 17.505012] hrtimer: interrupt took 12997651 ns Mounting Kernel Debug File System... [ OK ] Created slice system-getty.slice. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on initctl Compatibility Named Pipe. Starting Apply Kernel Variables... Starting Remount Root and Kernel File Systems... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Stopped target Initrd File Systems. [ OK ] Started Dispatch Password Requests to Console Directory Watch. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on udev Kernel Socket. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... Activating swap /dev/disk/by-label/SWAP... Mounting POSIX Message Queue File System... [ OK ] Reached target RPC Port Mapper. [ OK ] Started Journal Service. [ 18.622225] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Mounted Huge Pages File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Apply Kernel Variables. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Configure read-only root support... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 20.693922] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 21.821817] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 21.942720] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 25.197670] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 25.383771] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (9s / no limit) [** ] A start job is running for Configur…only root support (10s / no limit)[ 27.501614] Key type dns_resolver registered [*** ] A start job is running for Configur…only root support (10s / no limit)[ 27.944967] NFS: Registering the id_resolver key type [ 27.947924] Key type id_resolver registered [ 27.950657] Key type id_legacy registered [ *** ] A start job is running for Configur…only root support (11s / no limit) [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Load/Save Random Seed... Starting Create Volatile Files and Directories... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Login Service... Starting Restore /run/initramfs on shutdown... [ OK ] Started irqbalance daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg318-server login: [ 92.008694] libcfs: loading out-of-tree module taints kernel. [ 92.104683] Key type ._llcrypt registered [ 92.106782] Key type .llcrypt registered [ 92.259509] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_hostid [ 107.490339] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 108.744644] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 108.774533] alg: No test for adler32 (adler32-zlib) [ 110.364244] Lustre: Lustre: Build Version: 2.17.54_1_g05c2c2a [ 111.252842] LNet: Added LNI 192.168.203.118@tcp [8/256/0/180] [ 113.007150] Key type lgssc registered [ 114.458178] Lustre: Echo OBD driver; http://www.lustre.org/ [ 131.663794] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 166.620253] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 179.448322] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 179.475929] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 180.686682] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 180.720051] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 180.803403] Lustre: lustre-MDT0000: new disk, initializing [ 180.886331] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 180.902899] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 184.335481] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 197.774073] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 197.885358] Lustre: 6415:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 197.932310] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 197.937760] Lustre: Skipped 1 previous similar message [ 198.033348] Lustre: lustre-MDT0001: new disk, initializing [ 198.090706] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 198.140399] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 198.149908] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 202.005549] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 207.673443] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 215.564293] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 215.788983] Lustre: lustre-OST0000: new disk, initializing [ 215.792095] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 215.799286] Lustre: 8413:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 215.855073] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 221.672412] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 224.812545] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 224.823040] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 224.884828] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 235.869029] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 235.981904] Lustre: lustre-OST0001: new disk, initializing [ 235.985635] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 235.996210] Lustre: 9517:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 236.060656] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 241.179553] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 245.780445] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 245.797672] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 245.832323] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 253.138631] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 261.106369] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 268.154287] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing check_logdir /tmp/testlogs/ [ 273.193457] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing yml_node [ 278.019553] Lustre: DEBUG MARKER: Client: 2.17.54.1 [ 280.544493] Lustre: DEBUG MARKER: MDS: 2.17.54.1 [ 282.604703] Lustre: DEBUG MARKER: OSS: 2.17.54.1 [ 283.840640] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Tue Jun 9 13:57:33 EDT 2026 [ 298.309385] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 299.573316] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 315.873393] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 315.881961] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 315.893107] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 317.408111] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 317.412406] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 317.428520] Lustre: Skipped 2 previous similar messages [ 320.283236] Lustre: server umount lustre-MDT0000 complete [ 323.489174] LustreError: 7394:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781027894 with bad export cookie 15933036532057717771 [ 323.490755] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 323.496230] LustreError: 7394:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 327.647772] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 327.678271] LustreError: 8694:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 327.687147] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 327.703591] LustreError: 8694:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 327.707091] Lustre: Skipped 4 previous similar messages [ 328.672400] LustreError: 6420:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 328.695574] LustreError: 6420:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 329.798092] Lustre: server umount lustre-MDT0001 complete [ 332.929711] Lustre: server umount lustre-OST0000 complete [ 336.152820] Lustre: server umount lustre-OST0001 complete [ 342.111411] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_hostid [ 349.971155] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 378.287204] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 378.490770] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 378.517867] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 378.584343] Lustre: lustre-MDT0000: new disk, initializing [ 378.654818] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 378.668328] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 381.728087] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 390.762380] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 390.837633] Lustre: 14931:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 390.896746] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 390.902757] Lustre: Skipped 1 previous similar message [ 390.977807] Lustre: lustre-MDT0001: new disk, initializing [ 391.081067] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 391.123688] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 391.134944] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 394.238578] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 401.546662] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 404.714475] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 410.722338] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 411.031361] Lustre: lustre-OST0000: new disk, initializing [ 411.041133] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 411.048648] Lustre: 16769:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 411.176948] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 412.680899] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 412.692970] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 412.878129] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 416.191673] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 422.494353] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 425.134748] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 425.320870] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 428.307673] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 428.467873] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 433.121303] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 433.129976] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 433.135447] Lustre: Skipped 1 previous similar message [ 433.140537] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 433.144406] Lustre: Skipped 2 previous similar messages [ 438.755022] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 438.763602] Lustre: Skipped 2 previous similar messages [ 444.383275] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 444.486621] Lustre: server umount lustre-OST0000 complete [ 447.461922] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 447.466282] LustreError: Skipped 1 previous similar message [ 447.468922] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 447.478960] Lustre: Skipped 1 previous similar message [ 447.485365] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 447.489207] Lustre: Skipped 2 previous similar messages [ 461.279283] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 461.382438] Lustre: server umount lustre-MDT0000 complete [ 463.700201] LustreError: 14922:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781028034 with bad export cookie 15933036532057719311 [ 463.705713] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 463.711239] LustreError: 14922:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 463.867374] Lustre: server umount lustre-MDT0001 complete [ 467.193235] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 14:00:36 (1781028036) [ 471.769807] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 472.087194] LustreError: 18938:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 472.139521] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 474.848468] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 477.153177] LustreError: 18939:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 481.449635] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 484.440232] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 488.783338] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 491.691515] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 498.337831] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 502.230523] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 506.671282] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 520.139572] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 527.333716] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 527.342962] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 527.346298] Lustre: Skipped 2 previous similar messages [ 527.359163] Lustre: Skipped 8 previous similar messages [ 531.826610] Lustre: server umount lustre-MDT0000 complete [ 532.449353] LustreError: 18939:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 532.462952] LustreError: 18939:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 534.462191] LustreError: 18919:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781028105 with bad export cookie 15933036532057720193 [ 534.465922] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 534.470625] LustreError: 18919:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 534.676533] Lustre: server umount lustre-MDT0001 complete [ 547.748785] Lustre: server umount lustre-OST0000 complete [ 562.923843] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 571.685988] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 572.055642] LustreError: 23185:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 572.120782] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 572.124865] Lustre: Skipped 2 previous similar messages [ 575.609656] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 583.076556] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 586.688731] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 589.830684] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 595.086518] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 599.344623] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 605.706765] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 605.818067] Lustre: lustre-OST0001: new disk, initializing [ 605.823688] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 605.828869] Lustre: 25594:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 607.540189] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 607.551170] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 607.615716] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 609.616150] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 617.076863] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 619.616967] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 629.735095] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 629.741669] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 629.754520] Lustre: Skipped 1 previous similar message [ 629.759507] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 634.013299] LustreError: 26847:0:(ldlm_resource.c:1180:ldlm_resource_complain()) MGS: namespace resource [0x736d61726170:0x3:0x0].0x0 (ffff88d248fed300) refcount nonzero (2) after lock cleanup; forcing cleanup. [ 635.361493] LustreError: 23960:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 635.379268] LustreError: 23960:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 636.086941] Lustre: server umount lustre-MDT0000 complete [ 638.213281] LustreError: 24762:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781028209 with bad export cookie 15933036532057721040 [ 638.215823] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 638.220817] LustreError: 24762:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 638.442613] Lustre: server umount lustre-MDT0001 complete [ 651.355697] Lustre: server umount lustre-OST0000 complete [ 663.863575] Lustre: server umount lustre-OST0001 complete [ 672.835095] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 678.683543] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 678.936305] LustreError: 28623:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 681.248664] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 687.452994] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 690.332789] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 693.000678] Lustre: 29834:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 697.013988] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 701.354453] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 707.315695] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 707.500557] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 707.504979] Lustre: Skipped 6 previous similar messages [ 711.729128] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 717.779364] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 719.996847] Lustre: 31773:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 723.426947] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 723.436975] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 723.456496] Lustre: Skipped 3 previous similar messages [ 723.461118] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 723.466020] Lustre: Skipped 4 previous similar messages [ 729.674812] Lustre: server umount lustre-MDT0000 complete [ 732.328069] LustreError: 28605:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781028303 with bad export cookie 15933036532057722335 [ 732.331601] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 732.337133] LustreError: 28605:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 732.577406] Lustre: server umount lustre-MDT0001 complete [ 745.382225] Lustre: server umount lustre-OST0000 complete [ 757.987723] Lustre: server umount lustre-OST0001 complete [ 763.545559] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 14:05:33 (1781028333) [ 781.796942] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 788.901355] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 789.184200] LustreError: 34555:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 789.197450] LustreError: 34555:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 791.866798] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 798.777569] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 801.716639] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 804.178597] Lustre: 35764:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 808.539707] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 812.724875] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 818.851186] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 823.372334] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 828.917689] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 831.094163] Lustre: 37704:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 834.535858] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 834.547589] Lustre: Skipped 6 previous similar messages [ 839.645367] LustreError: 37856:0:(ldlm_resource.c:1180:ldlm_resource_complain()) MGS: namespace resource [0x736d61726170:0x3:0x0].0x0 (ffff88d24df29c00) refcount nonzero (1) after lock cleanup; forcing cleanup. [ 839.713324] Lustre: server umount lustre-MDT0000 complete [ 841.713396] LustreError: 34537:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781028412 with bad export cookie 15933036532057723441 [ 841.717122] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 841.719386] LustreError: 34537:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 841.875442] Lustre: server umount lustre-MDT0001 complete [ 853.005415] Lustre: server umount lustre-OST0000 complete [ 865.760299] Lustre: server umount lustre-OST0001 complete [ 871.117312] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 14:07:20 (1781028440) [ 881.994257] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 888.360648] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 891.070375] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 896.702654] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 899.050738] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 900.988738] Lustre: 41039:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 904.456065] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 908.228610] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 913.904379] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 917.172818] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 922.377457] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 925.076743] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 925.081047] Lustre: Skipped 1 previous similar message [ 927.713784] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 927.724485] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 927.729172] Lustre: Skipped 8 previous similar messages [ 933.250985] Lustre: server umount lustre-MDT0000 complete [ 934.580451] LustreError: 41586:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.203.18@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 934.602044] LustreError: 41586:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 14 previous similar messages [ 935.425559] LustreError: 42981:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781028506 with bad export cookie 15933036532057724540 [ 935.431389] LustreError: 42981:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 935.433308] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 941.739875] Lustre: server umount lustre-MDT0001 complete [ 946.665855] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 960.480305] LustreError: 3515:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff88d243074700 x1867542883036032/t0(0) o250->MGC192.168.203.118@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 962.882664] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 968.333447] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 968.643705] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 968.647402] Lustre: Skipped 9 previous similar messages [ 970.742208] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 970.826967] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 975.518981] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 976.743689] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 979.101407] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 980.173041] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 984.033096] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 984.042493] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 984.050557] Lustre: Skipped 6 previous similar messages [ 989.288425] Lustre: server umount lustre-MDT0000 complete [ 991.233700] LustreError: 39809:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781028562 with bad export cookie 15933036532057725667 [ 991.234547] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 991.241257] LustreError: 39809:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 991.378878] Lustre: server umount lustre-MDT0001 complete [ 1003.533113] Lustre: server umount lustre-OST0000 complete [ 1015.191501] Lustre: server umount lustre-OST0001 complete [ 1020.130127] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 14:09:49 (1781028589) [ 1028.942626] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 1034.190599] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1036.587723] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 1042.317539] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1044.757246] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 1046.711192] Lustre: 48651:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1046.718585] Lustre: 48651:0:(mgs_llog.c:1447:mgs_modify_param()) Skipped 1 previous similar message [ 1050.147368] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1053.383112] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 1058.636166] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1063.115074] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 1068.149611] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1070.420910] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1078.242470] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1081.722046] Lustre: server umount lustre-MDT0000 complete [ 1083.715167] LustreError: 47418:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781028654 with bad export cookie 15933036532057726458 [ 1083.716916] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1083.723092] LustreError: 47418:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1083.872966] Lustre: server umount lustre-MDT0001 complete [ 1096.230256] Lustre: server umount lustre-OST0000 complete [ 1108.465712] Lustre: server umount lustre-OST0001 complete [ 1113.398256] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 14:11:22 (1781028682) [ 1114.885097] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1115.927186] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 14:11:25 (1781028685) [ 1122.918810] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_hostid [ 1127.911177] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 1140.690548] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1140.860698] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1140.906297] Lustre: lustre-MDT0000: new disk, initializing [ 1140.970259] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1143.146347] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 1148.112959] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1151.395593] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1151.524551] Lustre: 54680:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1153.114408] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1153.131081] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1154.204062] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 1158.917256] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1159.881479] Lustre: DEBUG MARKER: create test files [ 1176.682883] Lustre: server umount lustre-OST0000 complete [ 1188.910477] Lustre: server umount lustre-MDT0000 complete [ 1193.417597] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing unload_modules_local [ 1194.807961] Key type lgssc unregistered [ 1194.977926] LNet: 56225:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1194.981614] LNetError: 56225:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1194.993864] LNet: Removed LNI 192.168.203.118@tcp [ 1195.421127] Key type .llcrypt unregistered [ 1195.424864] Key type ._llcrypt unregistered [ 1210.217110] Key type ._llcrypt registered [ 1210.219564] Key type .llcrypt registered [ 1210.283218] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 1210.788376] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1210.802730] alg: No test for adler32 (adler32-zlib) [ 1211.747123] Lustre: Lustre: Build Version: 2.17.54_1_g05c2c2a [ 1211.882265] LNet: Added LNI 192.168.203.118@tcp [8/256/0/180] [ 1213.495163] Key type lgssc registered [ 1214.188785] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1218.476991] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1218.488957] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1219.738899] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1221.582333] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 1224.702264] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1228.051030] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1228.293607] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1229.350067] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1229.384993] Lustre: 58852:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1229.389496] Lustre: 58852:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1229.392910] Lustre: 58852:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1229.396246] Lustre: 58852:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1229.400627] Lustre: 58852:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1229.404387] Lustre: 58852:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1230.901268] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 1233.886387] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1237.429091] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1240.414629] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1244.640866] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1244.641704] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1244.654400] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1249.385031] Lustre: server umount lustre-OST0000 complete [ 1251.395364] Lustre: server umount lustre-MDT0000 complete [ 1256.568301] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing unload_modules_local [ 1257.937534] Key type lgssc unregistered [ 1258.072446] LNet: 60377:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1258.077128] LNetError: 60377:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1258.087374] LNet: Removed LNI 192.168.203.118@tcp [ 1258.436553] Key type .llcrypt unregistered [ 1258.438093] Key type ._llcrypt unregistered [ 1273.327869] Key type ._llcrypt registered [ 1273.329631] Key type .llcrypt registered [ 1273.377857] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 1273.926736] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1273.939870] alg: No test for adler32 (adler32-zlib) [ 1274.828849] Lustre: Lustre: Build Version: 2.17.54_1_g05c2c2a [ 1274.929825] LNet: Added LNI 192.168.203.118@tcp [8/256/0/180] [ 1276.519162] Key type lgssc registered [ 1277.002989] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1280.630791] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1280.651813] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1281.891583] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1283.964426] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 1287.557043] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1291.150354] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1291.328577] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1294.157457] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 1296.358571] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1296.390819] Lustre: 63104:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1296.395087] Lustre: 63104:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1296.398023] Lustre: 63104:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1296.400755] Lustre: 63104:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1296.404214] Lustre: 63104:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1296.406981] Lustre: 63104:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1297.104630] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1300.589461] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1306.593496] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1306.597357] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1306.611640] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1310.261884] Lustre: server umount lustre-OST0000 complete [ 1312.180883] Lustre: server umount lustre-MDT0000 complete [ 1316.906455] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing unload_modules_local [ 1318.159940] Key type lgssc unregistered [ 1318.324336] LNet: 64577:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1318.327358] LNetError: 64577:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1318.341722] LNet: Removed LNI 192.168.203.118@tcp [ 1318.702799] Key type .llcrypt unregistered [ 1318.705118] Key type ._llcrypt unregistered [ 1325.997199] Key type ._llcrypt registered [ 1325.998767] Key type .llcrypt registered [ 1326.044728] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_hostid [ 1333.729529] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 1334.146651] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1334.175742] alg: No test for adler32 (adler32-zlib) [ 1335.092193] Lustre: Lustre: Build Version: 2.17.54_1_g05c2c2a [ 1335.225989] LNet: Added LNI 192.168.203.118@tcp [8/256/0/180] [ 1336.823146] Key type lgssc registered [ 1337.387892] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1353.893295] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1353.906492] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1355.031921] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1355.051427] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1355.092417] Lustre: lustre-MDT0000: new disk, initializing [ 1355.139856] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1355.149703] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1357.002215] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 1363.277809] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1363.343279] Lustre: 67669:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1363.384218] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1363.387862] Lustre: Skipped 1 previous similar message [ 1363.432251] Lustre: lustre-MDT0001: new disk, initializing [ 1363.490807] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1363.511633] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1363.516446] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1365.608572] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 1370.198325] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1372.223244] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1375.647355] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1375.800980] Lustre: lustre-OST0000: new disk, initializing [ 1375.803739] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1375.808330] Lustre: 69505:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1375.845415] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1378.821117] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 1383.595337] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1383.954411] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1383.965371] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1384.014310] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1385.466294] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1385.555767] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1387.306212] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1387.414733] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1389.028487] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1389.028814] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1389.029840] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1389.035554] Lustre: Skipped 1 previous similar message [ 1389.038452] LustreError: Skipped 1 previous similar message [ 1390.444865] Lustre: server umount lustre-OST0000 complete [ 1394.146196] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1394.146546] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1394.152813] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1394.152820] Lustre: Skipped 1 previous similar message [ 1394.161206] Lustre: Skipped 1 previous similar message [ 1398.300803] Lustre: server umount lustre-MDT0000 complete [ 1399.854884] LustreError: 67661:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781028970 with bad export cookie 9797465320845292622 [ 1399.857230] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1399.862912] LustreError: 67661:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1399.987110] Lustre: server umount lustre-MDT0001 complete [ 1403.905825] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 14:16:13 (1781028973) [ 1408.763659] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 1426.444819] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_hostid [ 1430.864202] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 1446.363420] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1446.507091] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1446.527945] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1446.565342] Lustre: lustre-MDT0000: new disk, initializing [ 1446.603313] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1446.613230] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1448.416622] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 1454.022879] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1454.078280] Lustre: 75363:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1454.110607] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1454.113052] Lustre: Skipped 1 previous similar message [ 1454.153907] Lustre: lustre-MDT0001: new disk, initializing [ 1454.206758] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1454.223738] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1454.229112] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1455.858752] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 1459.927124] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1461.595377] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1464.469472] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1464.591869] Lustre: lustre-OST0000: new disk, initializing [ 1464.594160] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1464.597709] Lustre: 77204:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1464.635751] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1467.038181] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 1471.451898] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1473.335490] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1474.054851] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1474.060232] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1474.113323] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1474.464441] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 1 sec [ 1476.240402] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1476.335468] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1479.138697] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1479.139919] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1479.144968] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1479.149692] Lustre: Skipped 1 previous similar message [ 1479.152340] Lustre: Skipped 1 previous similar message [ 1483.364977] Lustre: server umount lustre-OST0000 complete [ 1485.280788] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1485.284515] LustreError: Skipped 1 previous similar message [ 1485.286867] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1485.293629] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1485.296562] Lustre: Skipped 1 previous similar message [ 1489.379491] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1489.379746] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1489.393133] Lustre: Skipped 1 previous similar message [ 1491.237560] Lustre: server umount lustre-MDT0000 complete [ 1492.895274] LustreError: 75355:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781029064 with bad export cookie 9797465320845293504 [ 1492.897950] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1492.900087] LustreError: 75355:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1493.027975] Lustre: server umount lustre-MDT0001 complete [ 1497.041529] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 14:17:46 (1781029066) [ 1500.153620] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1500.395451] LustreError: 79519:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1500.405201] LustreError: 79519:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 1500.427203] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1502.093947] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 1505.759901] LustreError: 79520:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1506.027211] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1507.734758] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 1510.250949] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1511.686142] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1514.416045] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1517.030408] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 1519.903475] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1521.452978] Lustre: 81992:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting d99cc962-9063-48d8-b903-8b57f6358d2f at adminstrative request [ 1521.634443] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1521.641192] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 1531.873204] Lustre: MGS: Client d99cc962-9063-48d8-b903-8b57f6358d2f (at 0@lo) reconnecting [ 1531.880297] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x87f796ccd4313b24 to 0x87f796ccd4313daf [ 1531.886646] Lustre: MGC192.168.203.118@tcp: Connection restored to 0@lo (at 0@lo) [ 1554.120651] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1554.194358] Lustre: lustre-OST0001: new disk, initializing [ 1554.196265] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1554.199571] Lustre: 82388:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1554.236169] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1554.240210] Lustre: Skipped 2 previous similar messages [ 1556.229106] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 1560.011372] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1560.585045] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1560.589925] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1560.637234] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1565.667677] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1565.668296] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1565.671081] LustreError: Skipped 1 previous similar message [ 1565.675793] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 1565.679780] Lustre: Skipped 1 previous similar message [ 1567.241450] Lustre: server umount lustre-OST0001 complete [ 1570.789460] LustreError: 81208:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1570.798869] LustreError: 81208:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1575.907443] LustreError: 81208:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1575.913421] LustreError: 81208:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1581.027125] LustreError: 81206:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1581.031842] LustreError: 81206:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1583.071125] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1583.125609] Lustre: server umount lustre-OST0000 complete [ 1586.144677] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1586.145754] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1586.149080] Lustre: Skipped 4 previous similar messages [ 1586.152035] Lustre: Skipped 6 previous similar messages [ 1590.686307] Lustre: server umount lustre-MDT0000 complete [ 1591.265844] LustreError: 79520:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1591.276823] LustreError: 79520:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1592.237986] LustreError: 79499:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781029163 with bad export cookie 9797465320845295023 [ 1592.238844] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1592.242729] LustreError: 79499:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1592.385764] Lustre: server umount lustre-MDT0001 complete [ 1595.806145] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 14:19:25 (1781029165) [ 1596.389074] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 1597.084270] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 14:19:26 (1781029166) [ 1597.690397] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 1598.411284] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 14:19:28 (1781029168) [ 1599.045451] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 1599.749831] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 14:19:29 (1781029169) [ 1600.405202] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 1601.138385] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 14:19:31 (1781029171) [ 1601.771959] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 1603.906724] loop: module loaded [ 1603.910754] loop0: detected capacity change from 0 to 4294967296 [ 1603.924835] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 1604.495618] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 1605.507724] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 1607.022620] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 1617.188224] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1617.203946] LustreError: 86422:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 1617.206603] LustreError: 86422:0:(obd_mount.c:250:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 1617.209285] LustreError: 86422:0:(tgt_mount.c:2534:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 1617.212427] LustreError: 86422:0:(super25.c:184:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 1624.077540] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_hostid [ 1628.186297] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 1641.603434] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1641.695950] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1641.708506] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1641.741397] Lustre: lustre-MDT0000: new disk, initializing [ 1641.763906] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1641.770044] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1643.173508] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 1648.108425] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1648.155223] Lustre: 89246:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1648.269110] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1649.788759] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 1653.831290] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1655.369677] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1658.048655] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1658.166371] Lustre: 91087:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1660.432645] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 1661.188046] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1664.685859] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1666.555908] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1666.638772] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1668.319311] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1668.419245] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1671.649768] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1671.650429] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1671.653117] LustreError: Skipped 1 previous similar message [ 1671.654112] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1671.658854] Lustre: Skipped 1 previous similar message [ 1675.360622] Lustre: server umount lustre-OST0000 complete [ 1683.043738] Lustre: server umount lustre-MDT0000 complete [ 1684.439494] LustreError: 89240:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781029255 with bad export cookie 9797465320845295646 [ 1684.441721] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1684.446364] LustreError: 89240:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1684.582346] Lustre: server umount lustre-MDT0001 complete [ 1688.084281] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 14:20:57 (1781029257) [ 1690.621148] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1690.770220] LustreError: 93396:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1692.213748] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 1695.808510] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1697.345902] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 1699.653769] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1700.983760] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1703.411833] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1705.515256] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 1707.775314] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1709.187522] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1716.385591] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 7 sec [ 1717.819278] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1717.907203] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1720.296094] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1720.356260] Lustre: lustre-OST0001: new disk, initializing [ 1720.358508] Lustre: Skipped 2 previous similar messages [ 1720.360828] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1720.363670] Lustre: Skipped 3 previous similar messages [ 1720.366727] Lustre: 96586:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1722.336598] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 1726.050325] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1726.974157] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1726.977051] Lustre: Skipped 2 previous similar messages [ 1726.980151] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1726.985599] Lustre: Skipped 1 previous similar message [ 1727.016990] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1727.590713] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 1727.675538] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1729.094362] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 1729.174511] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1739.693460] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 1746.333904] Lustre: Failing over lustre-MDT0000 [ 1746.681483] Lustre: server umount lustre-MDT0000 complete [ 1747.427585] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1747.429104] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1747.430425] LustreError: Skipped 1 previous similar message [ 1747.430664] LustreError: 93391:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1747.436093] Lustre: Skipped 5 previous similar messages [ 1747.438642] LustreError: 93391:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 1761.700605] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 1761.702204] LDISKFS-fs (dm-0): recovery complete [ 1761.704717] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1761.743209] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1761.870758] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 1763.229874] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 1766.579648] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 1766.580208] Lustre: 93392:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1766.889085] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1766.909169] LustreError: 99306:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1767.223114] LustreError: 99306:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1767.226596] Lustre: 99306:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1767.230544] Lustre: 99306:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 7 previous similar messages [ 1767.546282] LustreError: 99306:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1767.550559] LustreError: 99306:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 1 previous similar message [ 1767.863186] LustreError: 99306:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1767.866630] LustreError: 99306:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 1 previous similar message [ 1768.495936] Lustre: 99306:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1768.499488] Lustre: 99306:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 3 previous similar messages [ 1768.817362] LustreError: 99306:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1768.820637] LustreError: 99306:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 3 previous similar messages [ 1769.135120] LustreError: 99306:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1769.138542] LustreError: 99306:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 3 previous similar messages [ 1770.705177] Lustre: 99306:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1770.710000] Lustre: 99306:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 6 previous similar messages [ 1771.023848] LustreError: 99306:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1771.027034] LustreError: 99306:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 6 previous similar messages [ 1771.343133] LustreError: 99306:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1771.346359] LustreError: 99306:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 6 previous similar messages [ 1774.783900] Lustre: 99306:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1774.787577] Lustre: 99306:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 12 previous similar messages [ 1775.104978] LustreError: 99306:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1775.107524] LustreError: 99306:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 12 previous similar messages [ 1775.423116] LustreError: 99306:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1775.425926] LustreError: 99306:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 12 previous similar messages [ 1782.920249] Lustre: 99306:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1782.925591] Lustre: 99306:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 25 previous similar messages [ 1783.241877] LustreError: 99306:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1783.246267] LustreError: 99306:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 25 previous similar messages [ 1783.559055] LustreError: 99306:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1783.562756] LustreError: 99306:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 25 previous similar messages [ 1799.167935] Lustre: 99306:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1799.174125] Lustre: 99306:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 51 previous similar messages [ 1799.488930] LustreError: 99306:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1799.492684] LustreError: 99306:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 51 previous similar messages [ 1799.807102] LustreError: 99306:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1799.809425] LustreError: 99306:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 51 previous similar messages [ 1826.023118] Lustre: 99306:0:(ldlm_lib.c:2380:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 1826.027194] Lustre: 99306:0:(ldlm_lib.c:2390:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 1826.029748] Lustre: 99306:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 0cfe9a13-3be7-4eb6-a365-3580ec03c044@192.168.203.18@tcp [ 1826.033872] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 1826.037127] Lustre: 99306:0:(ldlm_lib.c:1900:abort_req_replay_queue()) @@@ aborted: req@000000004251d7e8 x1867544157819648/t0(8589934786) o35->0cfe9a13-3be7-4eb6-a365-3580ec03c044@192.168.203.18@tcp:262/0 lens 392/0 e 0 to 0 dl 1781029407 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 1826.047966] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 1826.052829] Lustre: lustre-MDT0000: Denying connection for new client 0cfe9a13-3be7-4eb6-a365-3580ec03c044 (at 192.168.203.18@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 1826.057025] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 1826.069309] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1826.071896] Lustre: Skipped 2 previous similar messages [ 1826.074499] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 1826.090256] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:82 to 0x280000401:577) [ 1826.090281] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:49 to 0x2c0000401:545) [ 1831.399383] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1831.401955] Lustre: Skipped 5 previous similar messages [ 1833.145709] Lustre: server umount lustre-OST0000 complete [ 1834.464960] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1836.515235] LustreError: 95089:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1836.519793] LustreError: 95089:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 15 previous similar messages [ 1840.529858] Lustre: server umount lustre-OST0001 complete [ 1848.028845] Lustre: server umount lustre-MDT0000 complete [ 1849.353123] LustreError: 93424:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781029420 with bad export cookie 9797465320845346844 [ 1849.355991] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1849.357685] LustreError: 93424:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1849.460444] Lustre: server umount lustre-MDT0001 complete [ 1852.494911] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 14:23:42 (1781029422) [ 1855.053176] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1855.056643] Lustre: *** cfs_fail_loc=197, val=0*** [ 1870.495625] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 1875.615449] LustreError: 101305:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.118@tcp: failed processing log, type 4: rc = -110 [ 1880.742223] Lustre: *** cfs_fail_loc=197, val=0*** [ 1880.744040] Lustre: Skipped 65 previous similar messages [ 1901.215254] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1901.218157] Lustre: Skipped 7 previous similar messages [ 1903.116487] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 1905.385020] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1906.154067] Lustre: server umount lustre-OST0000 complete [ 1909.155449] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 14:24:39 (1781029479) [ 1922.942553] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1924.466917] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 1928.133115] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1930.189193] LustreError: 102889:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 1930.192897] LustreError: 102889:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 1930.195749] LustreError: 102889:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -5 [ 1930.311288] Lustre: server umount lustre-MDT0000 complete [ 1941.631456] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_hostid [ 1945.230642] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 1958.650881] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1958.756335] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1958.772098] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1958.812620] Lustre: lustre-MDT0000: new disk, initializing [ 1958.847118] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1960.416992] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 1965.651208] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1965.707394] Lustre: 106956:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1965.830367] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1967.418742] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 1971.448851] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1972.861855] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1975.463242] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1975.564343] Lustre: 108793:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1977.603453] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 1981.529799] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1983.165249] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1983.523316] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1984.283085] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 1 sec [ 1985.600696] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1985.678697] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1988.576802] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1988.578887] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1988.582842] LustreError: Skipped 2 previous similar messages [ 1988.588627] Lustre: Skipped 6 previous similar messages [ 1988.589668] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1988.595966] Lustre: Skipped 6 previous similar messages [ 1992.410974] Lustre: server umount lustre-OST0000 complete [ 1999.957512] Lustre: server umount lustre-MDT0000 complete [ 2001.271034] LustreError: 107781:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781029572 with bad export cookie 9797465320845357596 [ 2001.273925] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2001.275183] LustreError: 107781:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2001.379429] Lustre: server umount lustre-MDT0001 complete [ 2004.510786] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 14:26:14 (1781029574) [ 2007.971994] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing unload_modules_local [ 2009.134779] Key type lgssc unregistered [ 2009.282732] LNet: 111165:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2009.286437] LNetError: 111165:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2009.295379] LNet: Removed LNI 192.168.203.118@tcp [ 2009.655963] Key type .llcrypt unregistered [ 2009.657323] Key type ._llcrypt unregistered [ 2016.141071] Key type ._llcrypt registered [ 2016.142425] Key type .llcrypt registered [ 2016.186272] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_hostid [ 2023.646710] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 2024.055875] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2024.133070] alg: No test for adler32 (adler32-zlib) [ 2025.023879] Lustre: Lustre: Build Version: 2.17.54_1_g05c2c2a [ 2025.123330] LNet: Added LNI 192.168.203.118@tcp [8/256/0/180] [ 2026.727207] Key type lgssc registered [ 2027.154360] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2043.176211] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2043.186717] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2044.288682] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2044.303766] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2044.343042] Lustre: lustre-MDT0000: new disk, initializing [ 2044.381654] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2044.390672] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2045.929829] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 2051.063695] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2051.176780] Lustre: lustre-OST0000: new disk, initializing [ 2051.179337] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2051.182074] Lustre: Skipped 1 previous similar message [ 2051.185113] Lustre: 115310:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2051.220275] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2053.315126] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 2054.157780] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 2054.162389] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 2054.178808] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 2058.388705] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2058.457518] Lustre: lustre-OST0001: new disk, initializing [ 2058.460319] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2058.463528] Lustre: 116205:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2058.500068] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2060.822304] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 2061.725894] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 2061.732781] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 2061.751222] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 2066.914858] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2066.917658] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2066.925570] Lustre: Skipped 1 previous similar message [ 2066.930575] Lustre: Skipped 1 previous similar message [ 2071.924329] Lustre: server umount lustre-MDT0000 complete [ 2074.421365] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 2076.190690] LustreError: 114463:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781029647 with bad export cookie 6388514366959935663 [ 2076.192913] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2076.196025] LustreError: 114463:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2076.233604] Lustre: server umount lustre-OST0000 complete [ 2077.773183] Lustre: server umount lustre-OST0001 complete [ 2081.171111] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 14:27:31 (1781029651) [ 2084.714782] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 2093.455151] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 14:27:43 (1781029663) [ 2108.817334] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_hostid [ 2112.462435] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 2126.307691] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2126.420730] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2126.434533] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2126.474099] Lustre: lustre-MDT0000: new disk, initializing [ 2126.507057] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2126.515524] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2127.974890] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 2133.022386] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2133.068328] Lustre: 122917:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2133.095465] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2133.097597] Lustre: Skipped 1 previous similar message [ 2133.135782] Lustre: lustre-MDT0001: new disk, initializing [ 2133.173071] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2133.187808] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2133.191495] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2134.693789] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 2138.558793] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2139.953731] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2142.746119] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2142.858480] Lustre: lustre-OST0000: new disk, initializing [ 2142.860962] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2142.864758] Lustre: 124757:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2142.898215] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2145.116934] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 2148.863355] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2148.868748] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2148.910683] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2149.147480] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2150.788454] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2150.878572] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2152.397121] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2152.480719] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2153.957184] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2153.957386] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2153.958203] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2153.963822] Lustre: Skipped 1 previous similar message [ 2155.282541] Lustre: server umount lustre-OST0000 complete [ 2159.076658] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2159.076973] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2159.088365] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2159.091895] LustreError: Skipped 1 previous similar message [ 2159.094301] Lustre: Skipped 1 previous similar message [ 2162.778381] Lustre: server umount lustre-MDT0000 complete [ 2164.293580] LustreError: 122909:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781029735 with bad export cookie 6388514366959936874 [ 2164.296951] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2164.299308] LustreError: 122909:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2164.430586] Lustre: server umount lustre-MDT0001 complete [ 2167.700659] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 14:28:57 (1781029737) [ 2170.396062] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2170.594309] LustreError: 127064:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2170.603225] LustreError: 127064:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2170.626279] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2172.152735] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 2175.771279] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2177.196234] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 2179.256429] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2180.421106] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2182.762465] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2184.900883] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 2187.218608] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2199.731135] Lustre: *** cfs_fail_loc=159, val=0*** [ 2216.131705] Lustre: lustre-MDT0000: Client 64edaf49-49e7-447c-b54a-71b5e5865e24 (at 192.168.203.18@tcp) reconnecting [ 2216.661679] Lustre: *** cfs_fail_loc=159, val=0*** [ 2216.662959] Lustre: Skipped 5 previous similar messages [ 2233.025938] Lustre: lustre-MDT0000: Client 64edaf49-49e7-447c-b54a-71b5e5865e24 (at 192.168.203.18@tcp) reconnecting [ 2251.747077] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2251.747524] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2251.748200] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2251.751737] Lustre: Skipped 2 previous similar messages [ 2251.755679] LustreError: Skipped 1 previous similar message [ 2254.096431] Lustre: server umount lustre-OST0000 complete [ 2256.864655] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2256.872655] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2256.875603] Lustre: Skipped 2 previous similar messages [ 2261.588896] Lustre: server umount lustre-MDT0000 complete [ 2261.983917] LustreError: 129425:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2261.989407] LustreError: 129425:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2263.023173] LustreError: 127045:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781029834 with bad export cookie 6388514366959937756 [ 2263.026197] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2263.029296] LustreError: 127045:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2263.156552] Lustre: server umount lustre-MDT0001 complete [ 2266.660365] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing unload_modules_local [ 2267.629784] Key type lgssc unregistered [ 2267.746341] LNet: 130648:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2267.748791] LNetError: 130648:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2267.755288] LNet: Removed LNI 192.168.203.118@tcp [ 2268.072155] Key type .llcrypt unregistered [ 2268.075557] Key type ._llcrypt unregistered [ 2277.207022] Key type ._llcrypt registered [ 2277.207926] Key type .llcrypt registered [ 2277.244576] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 2277.625591] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2277.696718] alg: No test for adler32 (adler32-zlib) [ 2278.555962] Lustre: Lustre: Build Version: 2.17.54_1_g05c2c2a [ 2278.644966] LNet: Added LNI 192.168.203.118@tcp [8/256/0/180] [ 2280.223205] Key type lgssc registered [ 2280.618656] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2283.308562] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2283.314256] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2284.475822] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2285.774717] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 2289.060671] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2289.201881] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2290.559748] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 2292.730913] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2293.940883] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2296.233073] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2296.349371] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2298.234215] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 2300.472904] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2310.694206] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 2315.472793] Lustre: *** cfs_fail_loc=159, val=0*** [ 2330.812559] Lustre: lustre-MDT0000: Client 3225901d-9ad8-45a4-8ea0-398a90c084dd (at 192.168.203.18@tcp) reconnecting [ 2331.508775] Lustre: *** cfs_fail_loc=159, val=0*** [ 2346.690446] Lustre: lustre-MDT0001: Client 3225901d-9ad8-45a4-8ea0-398a90c084dd (at 192.168.203.18@tcp) reconnecting [ 2347.232995] Lustre: *** cfs_fail_loc=159, val=0*** [ 2347.234890] Lustre: Skipped 3 previous similar messages [ 2362.572447] Lustre: lustre-MDT0001: Client 3225901d-9ad8-45a4-8ea0-398a90c084dd (at 192.168.203.18@tcp) reconnecting [ 2363.794910] Lustre: 134880:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2364.709848] Lustre: *** cfs_fail_loc=159, val=0*** [ 2364.712212] Lustre: Skipped 5 previous similar messages [ 2380.992040] Lustre: lustre-MDT0000: Client aaf1138a-bc46-49ab-ac2d-8d004cb69191 (at 192.168.203.18@tcp) reconnecting [ 2381.562670] Lustre: *** cfs_fail_loc=159, val=0*** [ 2381.564066] Lustre: Skipped 13 previous similar messages [ 2397.895446] Lustre: lustre-MDT0000: Client aaf1138a-bc46-49ab-ac2d-8d004cb69191 (at 192.168.203.18@tcp) reconnecting [ 2398.483913] Lustre: 135171:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2401.759983] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2401.763957] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2401.769365] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2402.786488] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2402.786512] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2402.786604] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2406.170723] Lustre: server umount lustre-OST0000 complete [ 2407.904222] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2407.904913] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2407.909498] Lustre: Skipped 1 previous similar message [ 2407.911153] Lustre: Skipped 1 previous similar message [ 2413.024197] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2413.026786] Lustre: Skipped 2 previous similar messages [ 2413.718642] Lustre: server umount lustre-MDT0000 complete [ 2415.040489] LustreError: 132786:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781029986 with bad export cookie 12625509204157652633 [ 2415.044036] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2415.044770] LustreError: 132786:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2415.178147] Lustre: server umount lustre-MDT0001 complete [ 2418.958392] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing unload_modules_local [ 2420.017064] Key type lgssc unregistered [ 2420.155476] LNet: 136202:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2420.158501] LNetError: 136202:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2420.168339] LNet: Removed LNI 192.168.203.118@tcp [ 2420.488703] Key type .llcrypt unregistered [ 2420.490505] Key type ._llcrypt unregistered [ 2429.789219] Key type ._llcrypt registered [ 2429.790137] Key type .llcrypt registered [ 2429.826083] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 2430.187666] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2430.225776] alg: No test for adler32 (adler32-zlib) [ 2431.085156] Lustre: Lustre: Build Version: 2.17.54_1_g05c2c2a [ 2431.175434] LNet: Added LNI 192.168.203.118@tcp [8/256/0/180] [ 2432.751156] Key type lgssc registered [ 2433.091753] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2435.700343] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2435.705978] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2436.859822] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2438.067775] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 2441.302258] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2441.428883] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2442.691075] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 2444.779693] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2445.948757] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2448.256349] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2448.363605] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2450.280783] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 2451.430992] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 2452.335567] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2461.671290] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 2464.987721] Lustre: 139983:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2465.565466] Lustre: 140031:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2466.015728] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 2467.018111] Lustre: 140178:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 2467.285779] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 2467.288525] Lustre: Skipped 1 previous similar message [ 2467.555529] Lustre: 140276:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2471.905428] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2471.906961] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2471.907608] LustreError: Skipped 1 previous similar message [ 2471.913300] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2471.916260] Lustre: Skipped 1 previous similar message [ 2474.509528] Lustre: server umount lustre-OST0000 complete [ 2477.024872] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2477.028914] Lustre: Skipped 1 previous similar message [ 2477.030919] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2477.032529] Lustre: Skipped 1 previous similar message [ 2482.144212] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2482.147984] Lustre: Skipped 1 previous similar message [ 2487.264187] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2487.266814] Lustre: Skipped 1 previous similar message [ 2490.335118] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2490.400210] Lustre: server umount lustre-MDT0000 complete [ 2491.624232] LustreError: 137558:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781030062 with bad export cookie 148058205914642045 [ 2491.626634] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2491.627864] LustreError: 137558:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2491.735785] Lustre: server umount lustre-MDT0001 complete [ 2495.131095] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing unload_modules_local [ 2496.135688] Key type lgssc unregistered [ 2496.257363] LNet: 141316:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2496.259822] LNetError: 141316:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2496.271324] LNet: Removed LNI 192.168.203.118@tcp [ 2496.580722] Key type .llcrypt unregistered [ 2496.581740] Key type ._llcrypt unregistered [ 2505.355294] Key type ._llcrypt registered [ 2505.356279] Key type .llcrypt registered [ 2505.391397] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 2505.769343] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2505.784651] alg: No test for adler32 (adler32-zlib) [ 2506.638220] Lustre: Lustre: Build Version: 2.17.54_1_g05c2c2a [ 2506.722214] LNet: Added LNI 192.168.203.118@tcp [8/256/0/180] [ 2508.303143] Key type lgssc registered [ 2508.644410] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2511.244071] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2511.249903] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2512.414359] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2513.660060] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 2516.600719] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2516.727984] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2518.008034] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 2520.067957] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2521.250543] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2523.468092] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2523.573255] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2525.279852] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 2526.630154] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 2527.252516] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2537.895521] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 2544.816712] Lustre: *** cfs_fail_loc=159, val=0*** [ 2544.818880] Lustre: Skipped 1 previous similar message [ 2561.217893] Lustre: lustre-MDT0000: Client 1e47f479-cf7a-4931-915e-fbb1dddb16be (at 192.168.203.18@tcp) reconnecting [ 2577.887914] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2577.890296] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2577.894254] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2578.913245] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2578.913467] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2578.913841] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2578.923470] Lustre: Skipped 1 previous similar message [ 2582.212113] Lustre: server umount lustre-OST0000 complete [ 2583.519712] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2583.521556] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2583.525179] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2589.152210] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2589.154435] Lustre: Skipped 3 previous similar messages [ 2589.528121] Lustre: server umount lustre-MDT0000 complete [ 2590.762400] LustreError: 142671:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781030161 with bad export cookie 12889709810841003635 [ 2590.765929] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2590.766761] LustreError: 142671:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2590.872366] Lustre: server umount lustre-MDT0001 complete [ 2594.414573] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing unload_modules_local [ 2595.386853] Key type lgssc unregistered [ 2595.503374] LNet: 146164:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2595.505833] LNetError: 146164:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2595.517259] LNet: Removed LNI 192.168.203.118@tcp [ 2595.789247] Key type .llcrypt unregistered [ 2595.790172] Key type ._llcrypt unregistered [ 2604.963820] Key type ._llcrypt registered [ 2604.964965] Key type .llcrypt registered [ 2604.998265] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 2605.322260] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2605.362289] alg: No test for adler32 (adler32-zlib) [ 2606.208178] Lustre: Lustre: Build Version: 2.17.54_1_g05c2c2a [ 2606.293519] LNet: Added LNI 192.168.203.118@tcp [8/256/0/180] [ 2607.871190] Key type lgssc registered [ 2608.236823] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2610.818143] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2610.824742] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2611.977711] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2613.321196] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 2616.476275] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2616.613032] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2617.891881] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 2620.040040] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2621.266970] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2623.558062] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2623.667466] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2625.529234] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 2626.728670] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 2627.635154] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2637.986431] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 2650.598170] Lustre: lustre-OST0000: evicting 1861f7da-44a8-4ffc-9408-958b5cb4c3b4 (at 192.168.203.18@tcp) by administrative request [ 2650.601749] Lustre: Skipped 1 previous similar message [ 2678.240444] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2678.242918] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2678.246956] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2679.264492] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2679.266719] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2679.266770] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2680.650790] Lustre: server umount lustre-OST0000 complete [ 2683.359836] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2683.362103] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2683.366414] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2683.368013] Lustre: Skipped 1 previous similar message [ 2689.504663] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2689.507377] Lustre: Skipped 3 previous similar messages [ 2694.624257] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2694.626417] Lustre: Skipped 1 previous similar message [ 2696.159114] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2696.218397] Lustre: server umount lustre-MDT0000 complete [ 2697.544463] LustreError: 148309:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781030268 with bad export cookie 7791588939203004171 [ 2697.549289] LustreError: 148309:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2697.549826] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2697.686489] Lustre: server umount lustre-MDT0001 complete [ 2701.260557] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing unload_modules_local [ 2702.281841] Key type lgssc unregistered [ 2702.416336] LNet: 151413:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2702.420231] LNetError: 151413:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2702.427306] LNet: Removed LNI 192.168.203.118@tcp [ 2702.702178] Key type .llcrypt unregistered [ 2702.704203] Key type ._llcrypt unregistered [ 2727.187231] Key type ._llcrypt registered [ 2727.188450] Key type .llcrypt registered [ 2727.233657] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 2727.574509] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2727.625065] alg: No test for adler32 (adler32-zlib) [ 2728.491276] Lustre: Lustre: Build Version: 2.17.54_1_g05c2c2a [ 2728.579688] LNet: Added LNI 192.168.203.118@tcp [8/256/0/180] [ 2730.159141] Key type lgssc registered [ 2730.512954] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2733.080398] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2733.088674] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2734.263722] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2735.528454] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 2738.686069] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2738.817426] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2740.100814] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 2742.232453] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2743.456065] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2745.677438] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2745.797638] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2747.581368] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 2748.903082] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 2749.766322] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2759.142597] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 2764.258553] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2764.258715] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2764.263531] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2764.265045] LustreError: Skipped 1 previous similar message [ 2764.270069] Lustre: Skipped 1 previous similar message [ 2765.068597] Lustre: server umount lustre-OST0000 complete [ 2768.408490] Lustre: server umount lustre-MDT0000 complete [ 2769.376789] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2769.377267] LustreError: 153883:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2769.380238] Lustre: Skipped 2 previous similar messages [ 2769.384726] LustreError: 153883:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 2769.637390] LustreError: 153843:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781030340 with bad export cookie 2003495614730698820 [ 2769.640064] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2769.643032] LustreError: 153843:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2769.763196] Lustre: server umount lustre-MDT0001 complete [ 2773.483305] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing unload_modules_local [ 2774.471479] Key type lgssc unregistered [ 2774.598583] LNet: 157210:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2774.602952] LNetError: 157210:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2774.612391] LNet: Removed LNI 192.168.203.118@tcp [ 2774.954779] Key type .llcrypt unregistered [ 2774.956019] Key type ._llcrypt unregistered [ 2784.349709] Key type ._llcrypt registered [ 2784.351194] Key type .llcrypt registered [ 2784.398216] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 2784.781455] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2784.790850] alg: No test for adler32 (adler32-zlib) [ 2785.663117] Lustre: Lustre: Build Version: 2.17.54_1_g05c2c2a [ 2785.758937] LNet: Added LNI 192.168.203.118@tcp [8/256/0/180] [ 2787.343124] Key type lgssc registered [ 2787.715139] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2792.752117] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 14:39:22 (1781030362) [ 2798.089822] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_hostid [ 2801.158519] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 2813.640891] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2813.645772] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2814.736218] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2814.748381] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2814.793088] Lustre: lustre-MDT0000: new disk, initializing [ 2814.816152] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2814.822437] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2816.105674] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 2820.602685] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2820.641400] Lustre: 161392:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2820.663709] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2820.665576] Lustre: Skipped 1 previous similar message [ 2820.705269] Lustre: lustre-MDT0001: new disk, initializing [ 2820.738857] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2820.752512] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2820.755610] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2822.027190] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 2825.398305] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2826.571141] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2828.801896] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2828.906487] Lustre: lustre-OST0000: new disk, initializing [ 2828.909128] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2828.912934] Lustre: 163222:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2828.941128] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2829.563123] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2829.568340] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2829.613622] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2830.882710] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 2834.405737] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2840.034087] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2840.034457] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2840.035874] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2840.036672] LustreError: Skipped 1 previous similar message [ 2840.043260] Lustre: Skipped 1 previous similar message [ 2843.665720] Lustre: server umount lustre-OST0000 complete [ 2845.152694] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2845.153636] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2845.156415] Lustre: Skipped 1 previous similar message [ 2845.160863] Lustre: Skipped 2 previous similar messages [ 2850.272089] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2850.273967] Lustre: Skipped 1 previous similar message [ 2851.034732] Lustre: server umount lustre-MDT0000 complete [ 2852.299320] LustreError: 161382:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781030423 with bad export cookie 15313731096250021501 [ 2852.302279] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2852.305325] LustreError: 161382:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2852.442221] Lustre: server umount lustre-MDT0001 complete [ 2856.157910] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing unload_modules_local [ 2857.243228] Key type lgssc unregistered [ 2857.386444] LNet: 165046:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2857.389222] LNetError: 165046:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2857.397428] LNet: Removed LNI 192.168.203.118@tcp [ 2857.744276] Key type .llcrypt unregistered [ 2857.745796] Key type ._llcrypt unregistered [ 2866.859670] Key type ._llcrypt registered [ 2866.860883] Key type .llcrypt registered [ 2866.899977] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 2867.270952] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2867.319790] alg: No test for adler32 (adler32-zlib) [ 2868.179163] Lustre: Lustre: Build Version: 2.17.54_1_g05c2c2a [ 2868.264748] LNet: Added LNI 192.168.203.118@tcp [8/256/0/180] [ 2869.847145] Key type lgssc registered [ 2870.163940] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2872.506421] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2872.511635] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2873.646563] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2874.768235] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 2877.697843] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2877.800567] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2879.013130] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 2881.099824] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2882.274481] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2884.468307] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2884.586093] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2886.458842] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 2888.534726] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2897.895502] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 2901.537735] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2901.545498] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2901.551929] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2901.555745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2901.558934] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2901.897115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2902.214322] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2902.217201] Lustre: Skipped 1 previous similar message [ 2902.222362] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2902.225871] Lustre: Skipped 1 previous similar message [ 2902.227275] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2902.229138] Lustre: Skipped 1 previous similar message [ 2902.230377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2902.233431] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2902.235141] Lustre: Skipped 1 previous similar message [ 2902.519294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2902.824864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2903.131285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2903.432722] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2903.436174] Lustre: Skipped 3 previous similar messages [ 2903.442676] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2903.446390] Lustre: Skipped 3 previous similar messages [ 2903.447778] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2903.449578] Lustre: Skipped 3 previous similar messages [ 2903.450882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2903.455837] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2903.458748] Lustre: Skipped 3 previous similar messages [ 2903.746610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2904.036729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2904.324178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2904.673745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2904.989688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2905.284354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2905.570149] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2905.572601] Lustre: Skipped 6 previous similar messages [ 2905.578066] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2905.582412] Lustre: Skipped 6 previous similar messages [ 2905.584215] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2905.586275] Lustre: Skipped 6 previous similar messages [ 2905.587778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2905.591475] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2905.593914] Lustre: Skipped 6 previous similar messages [ 2905.892876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2906.192816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2906.480883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2906.786525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2907.099376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2907.388433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2907.691608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2907.990184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2908.300820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2908.608685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2908.906774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2909.216485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2909.544191] LustreError: 166439:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2909.547163] LustreError: 166439:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2909.548499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2909.852742] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2909.854882] Lustre: Skipped 13 previous similar messages [ 2909.859502] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2909.863550] Lustre: Skipped 13 previous similar messages [ 2909.865035] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2909.867058] Lustre: Skipped 13 previous similar messages [ 2909.868487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2909.871659] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2909.873562] Lustre: Skipped 13 previous similar messages [ 2910.162294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2910.474145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2910.788883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2911.085605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2911.382859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2911.674091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2912.009325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2912.317442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2912.648875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2912.963993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2913.319451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2913.629534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2913.960960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2914.266424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2914.638357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2914.961861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2915.268972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2915.574305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2915.888629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2916.197606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2916.499279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2916.796218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2917.122867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2917.417177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2917.714983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2918.012647] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2918.014818] Lustre: Skipped 25 previous similar messages [ 2918.020934] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2918.027476] Lustre: Skipped 25 previous similar messages [ 2918.030487] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2918.033984] Lustre: Skipped 25 previous similar messages [ 2918.036129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2918.041105] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2918.044911] Lustre: Skipped 25 previous similar messages [ 2918.351109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2918.637101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2918.934869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2919.225342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2919.537767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2919.829745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2920.145186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2920.443891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2920.755951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2921.045405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2921.349288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2921.651876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2921.976794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2922.266897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2922.572865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2922.879721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2923.216727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2923.539304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2923.856426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2924.137997] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2924.441900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2924.762759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2925.082720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2925.425531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2925.775454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2926.084420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2926.390285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2926.705687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2926.990682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2927.263586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2927.531345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2927.795970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2928.060229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2928.339301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2928.624336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2928.921685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2929.245610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2929.548433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2929.837212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2930.138304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2930.465430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2930.771388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2931.081518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2931.377238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2931.693120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2931.993877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2932.298765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2932.594722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2932.919801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2933.217161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2933.531255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2933.826919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2934.135084] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2934.137897] Lustre: Skipped 52 previous similar messages [ 2934.144252] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2934.147961] Lustre: Skipped 52 previous similar messages [ 2934.149406] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2934.151294] Lustre: Skipped 52 previous similar messages [ 2934.152593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2934.155907] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2934.157906] Lustre: Skipped 52 previous similar messages [ 2934.453610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2934.759414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2935.061804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2935.388784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2935.693959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2935.996324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2936.293578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2936.616290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2936.930493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2937.246142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2937.553090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2937.879906] LustreError: 166439:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2937.883134] LustreError: 166439:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2937.885087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2938.194689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2938.515473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2938.827465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2939.142425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2939.464388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2939.771207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2940.061192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2940.370275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2940.670645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2940.978090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2941.275846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2941.601658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2941.909628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2942.226152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2942.538867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2942.868329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2943.185113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2943.506596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2943.806169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2944.121763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2944.425870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2944.743940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2945.052597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2945.362736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2945.683179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2945.998349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2946.307669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2946.623948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2946.932251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2947.242227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2947.551186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2947.885730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2948.185832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2948.493494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2948.810651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2949.141204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2949.452030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2949.752117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2950.054769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2950.381883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2950.668417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2950.976806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2951.290841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2951.603635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2951.935274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2952.267695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2952.603779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2952.946815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2953.281147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2953.611532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2953.925635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2954.247870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2954.535152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2954.827113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2955.116626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2955.429759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2955.704589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2955.983602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2956.269018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2956.543621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2956.831760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2957.117240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2957.394584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2957.691677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2957.971503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2958.258462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2958.535810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2958.819835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2959.101272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2959.394503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2959.670262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2959.965960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2960.247536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2960.540848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2960.809705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2961.103458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2961.405390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2961.711864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2962.020968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2962.353073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2962.667308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2962.985502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2963.289801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2963.620627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2963.905804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2964.211027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2964.535214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2964.885716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2965.175694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2965.472813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2965.755521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2966.062324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2966.345815] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2966.347610] Lustre: Skipped 104 previous similar messages [ 2966.351774] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2966.354934] Lustre: Skipped 104 previous similar messages [ 2966.356845] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2966.359368] Lustre: Skipped 104 previous similar messages [ 2966.361061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2966.364062] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2966.365911] Lustre: Skipped 104 previous similar messages [ 2966.632810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2966.922884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2967.210800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2967.494943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2967.795100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2968.095330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2968.404976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2968.697046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2969.005131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2969.282436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2969.595312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2969.877611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2970.171750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2970.473763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2970.771792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2971.062744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2971.385044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2971.680818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2971.991465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2972.268418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2972.549191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2972.837229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2973.120148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2973.398119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2973.678849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2973.961970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2974.246751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2974.519531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2974.799498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2975.082062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2975.356413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2975.638767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2975.902992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2976.187508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2976.486684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2976.777248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2977.064258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2977.349859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2977.665595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2977.961819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2978.254195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2978.533315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2978.841804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2979.121617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2979.422546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2979.716696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2980.016840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2980.293431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2980.575127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2980.857477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2981.142510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2981.412063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2981.695412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2981.967626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2982.265840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2982.549043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2982.852343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2983.139676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2983.444984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2983.738092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2984.023202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2984.303404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2984.591185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2984.876108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2985.161679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2985.459188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2985.756316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2986.055048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2986.391509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2986.695670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2987.002834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2987.300250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2987.587748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2987.889864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2988.202756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2988.493354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2988.779284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2989.067762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2989.373741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2989.663076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2989.952540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2990.243210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2990.535313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2990.816960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2991.110296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2991.405036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2991.711507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2991.996867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2992.287401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2992.567929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2992.857361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2993.134981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2993.418061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2993.698797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2993.985963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2994.269314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2994.544932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2994.829702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2995.121202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2995.397619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2995.672823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2995.953195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2996.230933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2996.514154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2996.796434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2997.071284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2997.363777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2997.639551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2997.921987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2998.201025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2998.474250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2998.733527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2998.998493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2999.263277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2999.535755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2999.810061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3000.085719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3000.368233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3000.652621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3000.926707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3001.194156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3001.476927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3001.767991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3002.036596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3002.308551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3002.579704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3002.863034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3003.129122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3003.399165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3003.674352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3003.947479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3004.212231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3004.483452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3004.773153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3005.079905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3005.383672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3005.707475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3006.031221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3006.372931] LustreError: 166439:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3006.377113] LustreError: 166439:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3006.378167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3006.701770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3007.014537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3007.319156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3007.641086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3007.939239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3008.244088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3008.543072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3008.853022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3009.159243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3009.471580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3009.767639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3010.083015] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3010.369811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3010.663485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3010.968411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3011.281291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3011.581335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3011.882556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3012.192556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3012.524455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3012.836217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3013.140585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3013.447198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3013.763692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3014.084352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3014.399039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3014.700779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3015.019247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3015.322476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3015.655832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3015.969158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3016.308556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3016.622633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3016.925616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3017.217155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3017.518702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3017.790937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3018.070949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3018.363046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3018.676715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3018.983611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3019.278235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3019.579251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3019.898440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3020.198149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3020.490533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3020.790557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3021.102246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3021.401490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3021.714298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3022.010336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3022.306933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3022.605284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3022.909652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3023.222603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3023.537451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3023.856988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3024.171426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3024.465917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3024.790199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3025.083371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3025.397708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3025.721143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3026.042300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3026.357688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3026.674324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3026.984690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3027.301735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3027.601370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3027.898293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3028.196697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3028.519247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3028.820048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3029.118872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3029.412180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3029.716707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3030.006540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3030.296236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3030.590776] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3030.593633] Lustre: Skipped 217 previous similar messages [ 3030.600136] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3030.603811] Lustre: Skipped 217 previous similar messages [ 3030.605536] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3030.608131] Lustre: Skipped 217 previous similar messages [ 3030.610136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3030.613798] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3030.615550] Lustre: Skipped 217 previous similar messages [ 3030.913358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3031.202591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3031.485104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3031.767436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3032.063762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3032.344916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3032.624558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3032.893083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3033.176654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3033.447316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3033.724763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3034.001137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3034.286778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3034.560938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3034.838890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3035.115471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3035.397179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3035.677166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3035.950102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3036.221173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3036.503932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3036.779951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3037.046981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3037.319600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3037.593653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3037.865284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3038.138908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3038.407202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3038.675886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3038.948898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3039.223610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3039.487964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3039.762963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3040.031256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3040.312349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3040.586465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3040.876881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3041.154161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3041.432579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3041.703350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3041.995681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3042.268721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3042.558284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3042.837891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3043.119057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3043.392169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3043.666088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3043.940993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3044.215877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3044.489435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3044.770289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3045.050245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3045.345377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3045.620290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3045.897840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3046.179286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3046.468675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3046.758628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3047.050356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3047.338245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3047.667434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3047.956587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3048.241235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3048.521141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3048.825129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3049.110717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3049.394969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3049.679405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3049.983918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3050.271060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3050.560348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3050.849246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3051.140781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3051.419665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3051.714663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3052.008127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3052.310687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3052.599467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3052.888801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3053.175887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3053.495707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3053.782896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3054.075136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3054.351507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3054.639015] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3054.922948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3055.199259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3055.470946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3055.758569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3056.027641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3056.323316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3056.599684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3056.883469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3057.163618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3057.443088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3057.726907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3058.011611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3058.283238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3058.555197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3058.820028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3059.105444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3059.368652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3059.628466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3059.890704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3060.153138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3060.417885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3060.691024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3060.966638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3061.245393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3061.513555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3061.789404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3062.064170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3062.332731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3062.594630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3062.866930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3063.126222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3063.413183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3063.708988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3063.998675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3064.296884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3064.632643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3064.934589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3065.237688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3065.549831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3065.874850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3066.187625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3066.485928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3066.808626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3067.121870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3067.411618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3067.715946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3068.018513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3068.343085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3068.659754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3068.961689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3069.267277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3069.596509] LustreError: 166439:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3069.600428] LustreError: 166439:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3069.601716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3069.919037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3070.219032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3070.531688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3070.875021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3071.187958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3071.511601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3071.826691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3072.155077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3072.466405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3072.795850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3073.122774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3073.443155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3073.736392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3074.045557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3074.354163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3074.673081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3074.969501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3075.276596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3075.577992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3075.911363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3076.220764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3076.528058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3076.826081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3077.158476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3077.454944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3077.745592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3078.047092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3078.381749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3078.700872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3079.017622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3079.337955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3079.632712] LustreError: 166439:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3079.635843] LustreError: 166439:0:(osp_precreate.c:652:osp_precreate_send()) Skipped 1 previous similar message [ 3079.637654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3079.639127] LustreError: 166439:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3079.647514] LustreError: 166439:0:(osp_precreate.c:1356:osp_precreate_thread()) Skipped 1 previous similar message [ 3079.931573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3080.217413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3080.531151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3080.834833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3081.117343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3081.387938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3081.661285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3081.937264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3082.214670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3082.516910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3082.803451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3083.105051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3083.414192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3083.736662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3084.030443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3084.318923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3084.596895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3084.879389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3085.157876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3085.439979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3085.715150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3085.993729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3086.271157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3086.573509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3086.862111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3087.142048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3087.437920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3087.748402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3088.015667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3088.293349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3088.569992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3088.849323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3089.120930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3089.397768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3089.665744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3089.955614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3090.222918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3090.484975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3090.750411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3091.020018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3091.281236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3091.552566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3091.823020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3092.090796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3092.361982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3092.658151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3092.947554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3093.259832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3093.555681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3093.848343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3094.140406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3094.455750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3094.746354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3095.054199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3095.379206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3095.713894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3096.019303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3096.319933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3096.618477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3096.932624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3097.227269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3097.532947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3097.825569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3098.128215] LustreError: 166439:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3098.131204] LustreError: 166439:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3098.132521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3098.447068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3098.763889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3099.071799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3099.398297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3099.720194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3100.030790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3100.330911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3100.672276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3100.983227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3101.293945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3101.622938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3101.945147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3102.267909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3102.579470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3102.891745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3103.213940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3103.507481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3103.806635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3104.111644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3104.415922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3104.717851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3105.030642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3105.352491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3105.703946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3106.016908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3106.325425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3106.636495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3106.963373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3107.246056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3107.542074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3107.837024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3108.148439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3108.436240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3108.734614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3109.035282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3109.361351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3109.681332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3110.003847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3110.309872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3110.649950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3110.969676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3111.286762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3111.608323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3111.960403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3112.269834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3112.578948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3112.884762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3113.200039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3113.486464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3113.787613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3114.120905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3114.446501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3114.759936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3115.067396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3115.366809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3115.688382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3115.993993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3116.309244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3116.617870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3116.942215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3117.237658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3117.539754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3117.864863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3118.203456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3118.524843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3118.814995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3119.116991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3119.449630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3119.758786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3120.057780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3120.352829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3120.659852] LustreError: 166439:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3120.663069] LustreError: 166439:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3120.666669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3120.959351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3121.242643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3121.558058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3121.865990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3122.169898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3122.484037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3122.769490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3123.085498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3123.378635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3123.664295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3123.943510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3124.228705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3124.513734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3124.800367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3125.079571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3125.390674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3125.666365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3125.957278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3126.239792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3126.512971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3126.776196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3127.052835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3127.330762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3127.616555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3127.889396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3128.179665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3128.467542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3128.764126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3129.047886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3129.340109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3129.650143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3129.959840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3130.261754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3130.557100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3130.847648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3131.174114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3131.453835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3131.723682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3132.022667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3132.320849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3132.626752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3132.900378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3133.170785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3133.452993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3133.734056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3134.024985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3134.303898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3134.573295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3134.849394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3135.128581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3135.421102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3135.720683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3136.001894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3136.293820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3136.577774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3136.886610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3137.168831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3137.456047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3137.742443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3138.042967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3138.337615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3138.628789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3138.904675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3139.188470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3139.470713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3139.762661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3140.043245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3140.333039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3140.624495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3140.915815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3141.199762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3141.504297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3141.777809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3142.054613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3142.339487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3142.649078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3142.930133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3143.245726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3143.554077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3143.861239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3144.169731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3144.463183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3144.747604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3145.050035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3145.348641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3145.664498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3145.966105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3146.269659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3146.562714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3146.845189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3147.124613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3147.421970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3147.710292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3147.994726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3148.271512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3148.578606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3148.866112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3149.142524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3149.429525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3149.720707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3149.997239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3150.284430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3150.569957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3150.856869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3151.144942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3151.438031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3151.719477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3152.019904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3152.318806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3152.598898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3152.891315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3153.184413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3153.462887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3153.745167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3154.031989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3154.313754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3154.586224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3154.864786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3155.137464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3155.406136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3155.671752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3155.941286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3156.206284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3156.486892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3156.790529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3157.094685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3157.379584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3157.675073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3157.947646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3158.253152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3158.540110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3158.826669] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3158.829308] Lustre: Skipped 436 previous similar messages [ 3158.834663] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3158.838068] Lustre: Skipped 436 previous similar messages [ 3158.840033] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3158.843509] Lustre: Skipped 436 previous similar messages [ 3158.845655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3158.850473] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3158.852110] Lustre: Skipped 436 previous similar messages [ 3159.135716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3159.424161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3159.734572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3160.078401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3160.384064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3160.691055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3160.993704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3161.319368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3161.635542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3161.945715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3162.267685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3162.603674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3162.904422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3163.217287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3163.510418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3163.823482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3164.123845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3164.438418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3164.752231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3165.082682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3165.410246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3165.742107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3166.024034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3166.347227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3166.655060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3166.948815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3167.215660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3167.490077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3167.748513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3168.022882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3168.294491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3168.586085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3168.870274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3169.136403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3169.404201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3169.674143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3169.938975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3170.218895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3170.494093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3170.782726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3171.058546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3171.341879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3171.627187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3171.924675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3172.200299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3172.475530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3172.746044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3173.027321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3173.306889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3173.580858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3173.861357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3174.149182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3174.433445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3174.728063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3175.013366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3175.320879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3175.591855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3175.871380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3176.138030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3176.409625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3176.669981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3176.939682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3177.245806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3177.560353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3177.866769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3178.180920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3178.497972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3178.814169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3179.114719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3179.412905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3179.720912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3180.046835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3180.366064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3180.697983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3181.013437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3181.349091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3181.672399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3181.992836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3182.300763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3182.627086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3182.938763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3183.242778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3183.550476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3183.895145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3184.219587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3184.528891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3184.838161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3185.161089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3185.459198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3185.781587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3186.097958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3186.441532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3186.784551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3187.114130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3187.429850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3187.760728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3188.061096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3188.367609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3188.661375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3188.977145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3189.279550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3189.569945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3189.872623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3190.188795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3190.492159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3190.806963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3191.123744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3191.457232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3191.761374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3192.068118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3192.378382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3192.706478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3193.001994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3193.316619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3193.638711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3193.971221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3194.274717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3194.584409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3194.916132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3195.273097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3195.603691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3195.909792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3196.210905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3196.551296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3196.850724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3197.174482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3197.493984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3197.843867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3198.148747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3198.459804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3198.785023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3199.114877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3199.441098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3263.400992] LustreError: 168111:0:(ofd_dev.c:1782:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 3263.403972] LustreError: 166439:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 3263.406773] LustreError: 166439:0:(osp_precreate.c:652:osp_precreate_send()) Skipped 1 previous similar message [ 3321.826956] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3321.828910] Lustre: Skipped 1 previous similar message [ 3325.300265] Lustre: server umount lustre-OST0000 complete [ 3326.945046] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3326.947324] Lustre: Skipped 1 previous similar message [ 3332.064353] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3332.067443] Lustre: Skipped 1 previous similar message [ 3332.820618] Lustre: server umount lustre-MDT0000 complete [ 3334.109354] LustreError: 168132:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781030905 with bad export cookie 4625543399528427845 [ 3334.112449] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3334.112817] LustreError: 168132:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3334.240600] Lustre: server umount lustre-MDT0001 complete [ 3337.808129] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing unload_modules_local [ 3338.912759] Key type lgssc unregistered [ 3339.053374] LNet: 221240:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3339.056081] LNetError: 221240:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3339.063309] LNet: Removed LNI 192.168.203.118@tcp [ 3339.396925] Key type .llcrypt unregistered [ 3339.398663] Key type ._llcrypt unregistered [ 3348.764761] Key type ._llcrypt registered [ 3348.766608] Key type .llcrypt registered [ 3348.809965] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 3349.376995] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3349.391339] alg: No test for adler32 (adler32-zlib) [ 3350.261899] Lustre: Lustre: Build Version: 2.17.54_1_g05c2c2a [ 3350.338521] LNet: Added LNI 192.168.203.118@tcp [8/256/0/180] [ 3351.911138] Key type lgssc registered [ 3352.244435] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3354.574931] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3354.580052] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3355.727032] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3356.922831] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 3359.876216] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3360.001305] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3361.245110] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 3363.222132] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3364.423073] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3366.615327] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3366.729135] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3368.594798] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 3370.661569] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3380.080532] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 3383.872080] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3384.455946] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3384.462694] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3384.466904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3384.472440] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3384.472448] LustreError: 222633:0:(osp_precreate.c:1264:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 3385.953505] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 3385.954294] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3385.955979] LustreError: Skipped 3 previous similar messages [ 3385.961206] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3386.554409] Lustre: lustre-OST0000: Not available for connect from 192.168.203.18@tcp (stopping) [ 3386.556642] Lustre: Skipped 1 previous similar message [ 3390.434509] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3390.436432] Lustre: Skipped 1 previous similar message [ 3391.505995] Lustre: server umount lustre-OST0000 complete [ 3404.551066] LustreError: 222633:0:(osp_precreate.c:1264:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 3419.496523] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3419.611372] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3421.389910] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 3423.473553] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3428.096532] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 4 sec [ 3429.337634] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff923547024800.ost_server_uuid 50 [ 3429.867036] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff923547024800.ost_server_uuid in FULL state after 0 sec [ 3437.014080] Lustre: server umount lustre-OST0000 complete [ 3438.267855] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3438.269759] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3438.271628] Lustre: Skipped 2 previous similar messages [ 3444.438433] Lustre: server umount lustre-MDT0000 complete [ 3445.659187] LustreError: 222594:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781031016 with bad export cookie 17082582851307117197 [ 3445.661677] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3445.665583] LustreError: 222594:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3445.787555] Lustre: server umount lustre-MDT0001 complete [ 3449.429122] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing unload_modules_local [ 3450.416372] Key type lgssc unregistered [ 3450.536321] LNet: 227458:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3450.538761] LNetError: 227458:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3450.546341] LNet: Removed LNI 192.168.203.118@tcp [ 3450.849764] Key type .llcrypt unregistered [ 3450.851613] Key type ._llcrypt unregistered [ 3462.636488] Key type ._llcrypt registered [ 3462.637486] Key type .llcrypt registered [ 3462.681942] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 3462.997108] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3463.006909] alg: No test for adler32 (adler32-zlib) [ 3463.866490] Lustre: Lustre: Build Version: 2.17.54_1_g05c2c2a [ 3463.957047] LNet: Added LNI 192.168.203.118@tcp [8/256/0/180] [ 3465.543136] Key type lgssc registered [ 3465.906638] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3469.113122] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3469.119479] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3470.278277] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3471.595313] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 3474.858409] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3474.984479] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3476.323996] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 3477.667894] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 3479.926758] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3480.065496] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3481.127655] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 3482.304625] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 3485.590095] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3485.656575] Lustre: lustre-OST0001: new disk, initializing [ 3485.658244] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3485.660408] Lustre: 231548:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3485.689272] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3486.410203] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3486.413315] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3486.454152] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3487.616647] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 3493.565534] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3499.372905] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 3507.169698] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3507.172228] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3507.175422] Lustre: Skipped 3 previous similar messages [ 3507.179669] Lustre: Skipped 3 previous similar messages [ 3508.902410] Lustre: server umount lustre-MDT0000 complete [ 3510.152400] LustreError: 229128:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781031081 with bad export cookie 6386596550241218839 [ 3510.154901] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3510.155460] LustreError: 229128:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3512.295276] Lustre: server umount lustre-MDT0001 complete [ 3523.786097] Lustre: server umount lustre-OST0000 complete [ 3535.321401] Lustre: server umount lustre-OST0001 complete [ 3541.582194] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 3544.745485] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3544.878984] LustreError: 234672:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3544.886869] LustreError: 234672:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3544.900728] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3546.218210] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 3549.066560] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3550.314038] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 3551.521829] Lustre: 235873:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3553.508956] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3553.608439] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3553.610151] Lustre: Skipped 1 previous similar message [ 3555.233407] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 3555.683672] LustreError: 236226:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3555.685694] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49760) [ 3558.135748] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3559.921537] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 3563.216572] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3564.311261] Lustre: 237804:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3569.634851] Lustre: mylustre-MDT0000-lwp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3569.635830] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3569.637535] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3569.640644] Lustre: Skipped 3 previous similar messages [ 3573.521211] Lustre: server umount mylustre-MDT0000 complete [ 3574.648228] LustreError: 235444:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781031145 with bad export cookie 6386596550241221611 [ 3574.651821] LustreError: 235444:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3574.652208] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3574.749254] Lustre: server umount mylustre-MDT0001 complete [ 3586.182737] Lustre: server umount mylustre-OST0000 complete [ 3597.725535] Lustre: server umount mylustre-OST0001 complete [ 3604.719148] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 3608.133812] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3608.290390] LustreError: 240092:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3608.312903] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3608.315064] Lustre: Skipped 1 previous similar message [ 3609.647754] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 3612.859926] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3614.265202] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 3615.534288] Lustre: 241296:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3617.733253] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3618.917107] LustreError: 241649:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3618.921826] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49792) [ 3619.865454] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 3623.010618] LustreError: 241835:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3623.019225] LustreError: 241835:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3623.039161] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3625.020747] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 3628.588725] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3629.804788] Lustre: 243226:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3633.634165] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3633.635072] Lustre: tfs-MDT0000-lwp-OST0000: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3633.638626] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 3633.638635] Lustre: Skipped 3 previous similar messages [ 3633.641816] Lustre: Skipped 3 previous similar messages [ 3637.793326] Lustre: server umount tfs-MDT0000 complete [ 3638.754687] LustreError: 240093:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3638.759057] LustreError: 240093:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3638.984421] LustreError: 240069:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781031210 with bad export cookie 6386596550241223277 [ 3638.987125] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3638.987632] LustreError: 240069:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3639.094372] Lustre: server umount tfs-MDT0001 complete [ 3650.050763] Lustre: server umount tfs-OST0000 complete [ 3661.532287] Lustre: server umount tfs-OST0001 complete [ 3668.378512] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 3671.678929] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3671.845189] LustreError: 245454:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3671.869909] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3671.872132] Lustre: Skipped 3 previous similar messages [ 3673.226358] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 3676.324437] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3677.735802] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 3679.111137] Lustre: 246656:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3681.367088] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3682.534930] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49824) [ 3683.446805] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 3686.704567] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3688.576034] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 3692.083719] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3693.272593] Lustre: 248589:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3695.544390] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 14:54:25 (1781031265) [ 3707.360638] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3707.364701] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3707.369029] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3713.445292] Lustre: server umount lustre-MDT0000 complete [ 3714.723474] LustreError: 245430:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781031285 with bad export cookie 6386596550241224712 [ 3714.726460] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3714.729396] LustreError: 245430:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3715.552088] LustreError: 245451:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3715.552637] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3715.554022] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3715.554027] Lustre: Skipped 4 previous similar messages [ 3715.559101] LustreError: 245451:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 3715.570412] Lustre: Skipped 4 previous similar messages [ 3724.705135] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3724.707335] Lustre: Skipped 4 previous similar messages [ 3728.863140] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3728.947749] Lustre: server umount lustre-MDT0001 complete [ 3736.314414] Lustre: server umount lustre-OST0000 complete [ 3743.702865] Lustre: server umount lustre-OST0001 complete [ 3745.800548] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_hostid [ 3749.059168] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 3760.777061] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 3760.777061] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 3760.777061] [ 3760.785978] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 3760.869490] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3760.880358] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3760.918310] Lustre: lustre-MDT0000: new disk, initializing [ 3760.941064] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3760.943669] Lustre: Skipped 3 previous similar messages [ 3760.952245] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3762.380919] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 3766.939840] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 3766.939840] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 3766.939840] [ 3766.957447] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 3767.007377] Lustre: 251730:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3767.036390] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3767.039218] Lustre: Skipped 1 previous similar message [ 3767.083550] Lustre: lustre-MDT0001: new disk, initializing [ 3767.133758] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3767.138258] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3768.510415] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 3773.029041] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3773.130648] Lustre: lustre-OST0000: new disk, initializing [ 3773.132240] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3773.134640] Lustre: 253377:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3774.325476] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3774.329953] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3774.358845] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3774.968216] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 3779.456077] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3779.518296] Lustre: lustre-OST0001: new disk, initializing [ 3779.519892] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3779.522149] Lustre: 254276:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3781.424814] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 3781.448995] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3781.452160] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3781.495591] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3786.012752] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 14:55:55 (1781031355) [ 3789.707055] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 14:55:59 (1781031359) [ 3802.084472] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3802.089186] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3802.091012] Lustre: Skipped 1 previous similar message [ 3807.238759] Lustre: server umount lustre-OST0000 complete [ 3812.322972] LustreError: 253362:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3812.328098] LustreError: 253362:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 3814.684663] Lustre: server umount lustre-MDT0000 complete [ 3815.894348] LustreError: 251721:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781031387 with bad export cookie 6386596550241226042 [ 3815.896753] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3815.900067] LustreError: 251721:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3830.239160] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3830.331543] Lustre: server umount lustre-MDT0001 complete [ 3834.029806] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing unload_modules_local [ 3843.737729] Lustre: server umount lustre-OST0001 complete [ 3845.789826] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_hostid [ 3852.220258] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 3863.902313] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3863.997846] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3864.009972] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3864.051548] Lustre: lustre-MDT0000: new disk, initializing [ 3864.084719] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3865.363283] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 3869.702700] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3869.747895] Lustre: 258780:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3869.867204] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3871.235919] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 3874.668379] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3875.846263] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3878.091903] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3878.193036] Lustre: 260619:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3879.671957] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3879.708422] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3880.040331] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 3883.447524] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3890.145435] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3890.145993] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3890.149451] LustreError: Skipped 1 previous similar message [ 3890.156826] Lustre: Skipped 5 previous similar messages [ 3890.157658] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3890.162094] Lustre: Skipped 10 previous similar messages [ 3891.922644] Lustre: server umount lustre-OST0000 complete [ 3899.357085] Lustre: server umount lustre-MDT0000 complete [ 3900.577259] LustreError: 258773:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781031471 with bad export cookie 6386596550241227505 [ 3900.580338] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3900.580753] LustreError: 258773:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3900.703229] Lustre: server umount lustre-MDT0001 complete [ 3904.372298] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing unload_modules_local [ 3905.474870] Key type lgssc unregistered [ 3905.616463] LNet: 262296:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3905.619129] LNetError: 262296:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3905.626448] LNet: Removed LNI 192.168.203.118@tcp [ 3905.929108] Key type .llcrypt unregistered [ 3905.930385] Key type ._llcrypt unregistered [ 3915.878058] Key type ._llcrypt registered [ 3915.879113] Key type .llcrypt registered [ 3915.920373] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 3916.322512] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3916.331505] alg: No test for adler32 (adler32-zlib) [ 3917.226742] Lustre: Lustre: Build Version: 2.17.54_1_g05c2c2a [ 3917.319542] LNet: Added LNI 192.168.203.118@tcp [8/256/0/180] [ 3918.903117] Key type lgssc registered [ 3919.307491] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3921.893933] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3921.900158] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3923.048610] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3924.323102] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 3927.467858] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3927.594402] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3928.829826] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 3930.956974] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3932.147278] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3934.368705] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3934.471929] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3936.245385] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 3938.389137] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3938.759211] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 3942.882599] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3942.885210] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3942.888945] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3945.485466] Lustre: server umount lustre-OST0000 complete [ 3948.001082] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3948.001505] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3948.004183] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3948.007913] Lustre: Skipped 1 previous similar message [ 3953.120244] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3953.122292] Lustre: Skipped 2 previous similar messages [ 3958.240390] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3958.243346] Lustre: Skipped 1 previous similar message [ 3961.311099] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3961.370809] Lustre: server umount lustre-MDT0000 complete [ 3962.653115] LustreError: 263700:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781031533 with bad export cookie 10365678276096539277 [ 3962.656052] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3962.657855] LustreError: 263700:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3962.778118] Lustre: server umount lustre-MDT0001 complete [ 3966.575289] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing unload_modules_local [ 3967.620215] Key type lgssc unregistered [ 3967.743466] LNet: 267088:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3967.746921] LNetError: 267088:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3967.755309] LNet: Removed LNI 192.168.203.118@tcp [ 3968.050931] Key type .llcrypt unregistered [ 3968.052190] Key type ._llcrypt unregistered [ 3974.777234] Key type ._llcrypt registered [ 3974.778245] Key type .llcrypt registered [ 3974.816852] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 3975.241710] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3975.248059] alg: No test for adler32 (adler32-zlib) [ 3976.092640] Lustre: Lustre: Build Version: 2.17.54_1_g05c2c2a [ 3976.174423] LNet: Added LNI 192.168.203.118@tcp [8/256/0/180] [ 3977.759153] Key type lgssc registered [ 3978.107419] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3981.025489] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3981.033470] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3996.447369] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 4001.567238] LustreError: 268246:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.118@tcp: failed processing log, type 4: rc = -110 [ 4028.191147] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4030.038286] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 4032.184797] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4032.926556] Lustre: server umount lustre-OST0000 complete [ 4038.227819] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing unload_modules_local [ 4039.133876] Key type lgssc unregistered [ 4039.254274] LNet: 269791:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4039.256678] LNetError: 269791:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4039.268275] LNet: Removed LNI 192.168.203.118@tcp [ 4039.543484] Key type .llcrypt unregistered [ 4039.544714] Key type ._llcrypt unregistered [ 4053.032473] Key type ._llcrypt registered [ 4053.035167] Key type .llcrypt registered [ 4053.076729] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 4053.495168] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4053.512268] alg: No test for adler32 (adler32-zlib) [ 4054.375203] Lustre: Lustre: Build Version: 2.17.54_1_g05c2c2a [ 4054.460456] LNet: Added LNI 192.168.203.118@tcp [8/256/0/180] [ 4056.039117] Key type lgssc registered [ 4056.405118] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4060.625218] loop0: detected capacity change from 0 to 204800000 [ 4060.673590] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4060.677132] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4060.804699] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4061.136232] loop0: detected capacity change from 0 to 204800000 [ 4061.197907] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4061.201458] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4061.324019] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4061.667785] loop0: detected capacity change from 0 to 204800000 [ 4061.727697] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4061.732591] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4061.775135] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4062.155151] loop0: detected capacity change from 0 to 204800000 [ 4062.216630] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4062.608498] loop0: detected capacity change from 0 to 204800000 [ 4062.619801] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4063.995847] loop0: detected capacity change from 0 to 204800000 [ 4064.010915] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4065.440387] loop0: detected capacity change from 0 to 204800000 [ 4065.457655] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4066.854340] loop0: detected capacity change from 0 to 204800000 [ 4066.866564] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4068.294389] loop0: detected capacity change from 0 to 204800000 [ 4068.322846] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4068.334379] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4068.375705] Lustre: 273162:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4068.380807] Lustre: 273162:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 4068.386926] Lustre: 273162:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 4068.391464] Lustre: 273162:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 4068.395924] Lustre: 273162:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 4068.399928] Lustre: 273162:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 4068.405388] Lustre: 273162:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 4068.410428] Lustre: 273162:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 4068.415480] Lustre: 273162:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 4068.420481] Lustre: 273162:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 4068.426361] Lustre: 273162:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 4068.431972] Lustre: 273162:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 4068.437210] Lustre: 273162:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 4068.442424] Lustre: 273162:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 4071.069827] Lustre: server umount MGS complete [ 4071.386750] loop0: detected capacity change from 0 to 204800000 [ 4071.415563] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4071.633141] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4071.652879] Lustre: lustre-MDT0000: Aborting client recovery [ 4071.988264] loop1: detected capacity change from 0 to 204800000 [ 4072.018738] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4072.064239] Lustre: 273594:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4072.070885] Lustre: 273594:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 4072.076332] Lustre: 273594:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 4072.082433] Lustre: 273594:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 4072.088118] Lustre: 273594:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 4072.094354] Lustre: 273594:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 4072.099421] Lustre: 273594:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 4072.108129] Lustre: Found index 1 for lustre-MDT0001, updating log [ 4072.189215] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4072.223761] Lustre: lustre-MDT0001: Aborting client recovery [ 4072.558920] loop2: detected capacity change from 0 to 204800000 [ 4072.591740] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4072.623144] Lustre: 273674:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4072.626940] Lustre: 273674:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 4072.630699] Lustre: 273674:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 4072.633958] Lustre: 273674:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 4072.637940] Lustre: 273674:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 4072.641529] Lustre: 273674:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 4072.648571] Lustre: 273674:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 4072.704056] Lustre: Found index 0 for lustre-OST0000, updating log [ 4072.718840] Lustre: 273692:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4072.744933] Lustre: lustre-OST0000: Aborting recovery [ 4073.072815] loop3: detected capacity change from 0 to 204800000 [ 4073.108164] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4073.142063] Lustre: 273757:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4073.146446] Lustre: 273757:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 4073.150306] Lustre: 273757:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 4073.156371] Lustre: 273757:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 4073.160706] Lustre: 273757:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 4073.164834] Lustre: 273757:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 4073.170098] Lustre: 273757:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 4073.214352] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4073.217409] Lustre: Skipped 1 previous similar message [ 4078.567564] LustreError: 273538:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -108 [ 4081.634687] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 4081.635310] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 4081.648700] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 4081.651227] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 4082.658463] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 4082.658495] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 4083.872145] LustreError: 273505:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 242us: rc = -2 [ 4098.017074] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4098.017507] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4098.017646] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4098.021667] Lustre: Skipped 3 previous similar messages [ 4101.291334] Lustre: server umount lustre-MDT0000 complete [ 4101.616646] LustreError: 273487:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781031672 with bad export cookie 8563819454643732249 [ 4101.618789] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4101.620733] LustreError: 273487:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4103.135737] LustreError: 273989:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4103.135982] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4103.136342] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4103.136348] Lustre: Skipped 3 previous similar messages [ 4103.141041] LustreError: 273989:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 4103.144629] Lustre: Skipped 1 previous similar message [ 4106.143612] LustreError: 273503:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4106.143682] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4106.149957] LustreError: 273503:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4106.156342] Lustre: Skipped 2 previous similar messages [ 4107.838464] Lustre: server umount lustre-MDT0001 complete [ 4114.251122] Lustre: server umount lustre-OST0000 complete [ 4120.734580] Lustre: server umount lustre-OST0001 complete [ 4123.380561] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 15:01:33 (1781031693) [ 4131.889779] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_hostid [ 4134.930906] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 4146.986196] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4147.081232] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4147.092066] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4147.127591] Lustre: lustre-MDT0000: new disk, initializing [ 4147.151457] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4147.157332] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4148.456124] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 4152.945522] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4152.999984] Lustre: 277800:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4153.030432] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4153.032438] Lustre: Skipped 1 previous similar message [ 4153.071921] Lustre: lustre-MDT0001: new disk, initializing [ 4153.111374] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4153.126227] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4153.130343] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4154.517649] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 4158.116082] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4159.353692] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4161.626327] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4161.724834] Lustre: lustre-OST0000: new disk, initializing [ 4161.726818] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4161.729697] Lustre: 279634:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4161.733038] Lustre: 279634:0:(osd_compat.c:1352:osd_object_spec_find()) Skipped 1 previous similar message [ 4161.763269] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4161.961594] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4161.966042] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4162.010070] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4163.672703] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 4167.067578] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4169.477266] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 4177.083261] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 4185.450041] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 4189.956183] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 4218.340842] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4218.341908] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4218.343888] LustreError: Skipped 1 previous similar message [ 4218.350716] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4218.354343] Lustre: Skipped 1 previous similar message [ 4223.458097] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4223.460071] Lustre: Skipped 1 previous similar message [ 4232.159137] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4232.220813] Lustre: server umount lustre-OST0000 complete [ 4233.696499] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4233.697117] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4233.700323] Lustre: Skipped 2 previous similar messages [ 4233.702226] Lustre: Skipped 2 previous similar messages [ 4239.583561] Lustre: server umount lustre-MDT0000 complete [ 4240.863831] LustreError: 279633:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781031811 with bad export cookie 8563819454643734951 [ 4240.867703] LustreError: 279633:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4240.868156] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4241.003154] Lustre: server umount lustre-MDT0001 complete [ 4243.921329] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4245.258518] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 4247.316970] Lustre: server umount MGS complete [ 4251.156902] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4251.348316] LustreError: 283696:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4251.356583] LustreError: 283709:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 4251.380438] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4252.745033] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 4255.831041] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4257.310742] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 4259.466737] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4260.672396] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4262.953437] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4264.991355] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 4267.279742] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4297.189096] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4297.190126] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4297.190954] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4297.190960] Lustre: Skipped 2 previous similar messages [ 4297.191492] LustreError: Skipped 1 previous similar message [ 4297.203927] Lustre: Skipped 1 previous similar message [ 4302.926541] Lustre: server umount lustre-OST0000 complete [ 4307.424842] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4307.425190] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4310.305280] Lustre: server umount lustre-MDT0000 complete [ 4311.554756] LustreError: 284463:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781031882 with bad export cookie 8563819454643736652 [ 4311.557602] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4311.560455] LustreError: 284463:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4311.674453] Lustre: server umount lustre-MDT0001 complete [ 4315.463060] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing unload_modules_local [ 4316.340643] Key type lgssc unregistered [ 4316.462509] LNet: 287444:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4316.467590] LNetError: 287444:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4316.478429] LNet: Removed LNI 192.168.203.118@tcp [ 4316.874866] Key type .llcrypt unregistered [ 4316.878278] Key type ._llcrypt unregistered [ 4328.709329] Key type ._llcrypt registered [ 4328.710683] Key type .llcrypt registered [ 4328.751481] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_hostid [ 4334.864591] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 4335.175822] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4335.182477] alg: No test for adler32 (adler32-zlib) [ 4336.046325] Lustre: Lustre: Build Version: 2.17.54_1_g05c2c2a [ 4336.131949] LNet: Added LNI 192.168.203.118@tcp [8/256/0/180] [ 4337.719196] Key type lgssc registered [ 4338.181307] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4349.541795] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4349.548323] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4350.643762] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4350.657242] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4350.700065] Lustre: lustre-MDT0000: new disk, initializing [ 4350.726623] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4350.733771] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4352.155015] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 4356.805690] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4356.849318] Lustre: 291339:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4356.872316] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4356.874317] Lustre: Skipped 1 previous similar message [ 4356.913440] Lustre: lustre-MDT0001: new disk, initializing [ 4356.945637] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4356.958446] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4356.962454] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4358.268614] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 4361.705207] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4362.881265] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4365.107974] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4365.239039] Lustre: lustre-OST0000: new disk, initializing [ 4365.240783] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4365.243683] Lustre: 293177:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4365.278031] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4367.153749] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 4367.990806] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4367.994485] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4368.032821] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4370.562415] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4374.911426] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 4378.682859] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 4387.040299] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 4391.556992] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 4429.793408] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4429.794575] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4429.796156] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4429.798816] Lustre: Skipped 1 previous similar message [ 4431.575480] Lustre: server umount lustre-OST0000 complete [ 4433.887864] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4433.890415] LustreError: Skipped 1 previous similar message [ 4433.892153] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4433.898586] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4433.900939] Lustre: Skipped 1 previous similar message [ 4434.912175] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4434.912996] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4440.032094] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4440.034364] Lustre: Skipped 2 previous similar messages [ 4445.153067] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4445.156528] Lustre: Skipped 1 previous similar message [ 4447.199123] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4447.265804] Lustre: server umount lustre-MDT0000 complete [ 4448.467384] LustreError: 291330:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781032019 with bad export cookie 3186357355386886602 [ 4448.470721] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4448.473087] LustreError: 291330:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4448.583501] Lustre: server umount lustre-MDT0001 complete [ 4451.355474] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4452.682584] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 4454.631164] Lustre: server umount MGS complete [ 4458.438989] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4458.610489] LustreError: 297110:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 4458.638565] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4459.996548] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 4463.197861] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4463.325202] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4464.639954] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 4466.703790] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4467.875628] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4470.144752] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4472.139735] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 4474.322885] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4498.913940] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4498.914150] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4498.919598] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4498.921545] LustreError: Skipped 1 previous similar message [ 4498.924754] Lustre: Skipped 1 previous similar message [ 4502.673170] Lustre: server umount lustre-OST0000 complete [ 4504.032944] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4504.036318] Lustre: Skipped 2 previous similar messages [ 4518.367143] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4518.442615] Lustre: server umount lustre-MDT0000 complete [ 4519.393313] LustreError: 297098:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4519.398139] LustreError: 297098:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 4519.678355] LustreError: 297079:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781032090 with bad export cookie 3186357355386888429 [ 4519.682101] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4519.684333] LustreError: 297079:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4519.811944] Lustre: server umount lustre-MDT0001 complete [ 4523.289724] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing unload_modules_local [ 4524.282392] Key type lgssc unregistered [ 4524.420351] LNet: 300847:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4524.422672] LNetError: 300847:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4524.432387] LNet: Removed LNI 192.168.203.118@tcp [ 4524.729480] Key type .llcrypt unregistered [ 4524.730734] Key type ._llcrypt unregistered [ 4531.389174] Key type ._llcrypt registered [ 4531.390284] Key type .llcrypt registered [ 4531.429129] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_hostid [ 4537.302571] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 4537.618523] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4537.626184] alg: No test for adler32 (adler32-zlib) [ 4538.487853] Lustre: Lustre: Build Version: 2.17.54_1_g05c2c2a [ 4538.571974] LNet: Added LNI 192.168.203.118@tcp [8/256/0/180] [ 4540.151219] Key type lgssc registered [ 4540.500903] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4548.246522] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 4553.295286] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4553.301776] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4554.401820] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4554.416431] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4554.460144] Lustre: lustre-MDT0000: new disk, initializing [ 4554.482900] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4554.489255] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4555.817886] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 4560.437129] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4560.539741] Lustre: lustre-OST0000: new disk, initializing [ 4560.542628] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4560.545788] Lustre: Skipped 1 previous similar message [ 4560.548853] Lustre: 304852:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4560.576404] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4562.441619] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 4568.572659] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 4568.576166] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 4568.583910] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 4742.624201] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4742.627803] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4746.927597] Lustre: server umount lustre-MDT0000 complete [ 4748.236366] LustreError: 304007:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781032319 with bad export cookie 1675216977264461102 [ 4748.238909] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4748.241553] LustreError: 304007:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4748.300369] Lustre: server umount lustre-OST0000 complete [ 4756.641428] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_hostid [ 4759.831255] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 4772.439673] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4772.531690] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4772.542343] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4772.580593] Lustre: lustre-MDT0000: new disk, initializing [ 4772.603908] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4772.610634] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4773.901363] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 4778.255839] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4778.295486] Lustre: 309073:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4778.316344] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4778.319047] Lustre: Skipped 1 previous similar message [ 4778.358430] Lustre: lustre-MDT0001: new disk, initializing [ 4778.390873] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4778.401668] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4778.405794] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4779.699033] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 4783.120839] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4784.332993] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4786.545461] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4786.645886] Lustre: lustre-OST0000: new disk, initializing [ 4786.647589] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4786.650089] Lustre: 310910:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4786.677035] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4787.577381] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4787.582772] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4787.616102] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4788.586632] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 4792.081657] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4793.573074] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4793.650404] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4795.021780] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4795.102049] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4797.922003] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4797.922964] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4797.926505] LustreError: Skipped 1 previous similar message [ 4797.930430] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4797.933949] Lustre: Skipped 1 previous similar message [ 4803.043445] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4803.047343] Lustre: Skipped 1 previous similar message [ 4808.162372] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4808.164220] Lustre: Skipped 1 previous similar message [ 4810.207199] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4810.289970] Lustre: server umount lustre-OST0000 complete [ 4813.282674] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4813.283990] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4813.290370] Lustre: Skipped 2 previous similar messages [ 4823.521133] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4823.526554] Lustre: Skipped 4 previous similar messages [ 4826.079142] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4826.168172] Lustre: server umount lustre-MDT0000 complete [ 4827.549898] LustreError: 309065:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781032398 with bad export cookie 1675216977267401900 [ 4827.557463] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4827.559544] LustreError: 309065:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4827.704531] Lustre: server umount lustre-MDT0001 complete [ 4828.873386] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 4829.490542] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 15:13:19 (1781032399) [ 4831.768752] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4831.934214] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4833.219638] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 4836.658167] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4838.512453] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 4841.174341] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4842.355615] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4844.728807] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4846.741616] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 4848.864400] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4851.533840] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4851.610809] Lustre: lustre-OST0001: new disk, initializing [ 4851.612673] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4851.615056] Lustre: 315764:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4851.643649] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4851.645808] Lustre: Skipped 2 previous similar messages [ 4853.765814] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 4854.917097] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 4854.920316] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 4854.937122] LustreError: 316254:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 4855.970872] LustreError: 316254:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 4855.975086] LustreError: 316254:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 4856.998903] LustreError: 316259:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 4857.004060] LustreError: 316259:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 4857.955411] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4858.025504] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4859.042377] LustreError: 316259:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 4859.045117] LustreError: 316259:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 4859.211322] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff92354b072800.ost_server_uuid 50 [ 4859.760702] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff92354b072800.ost_server_uuid in FULL state after 0 sec [ 4861.216559] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff92354b072800.ost_server_uuid 50 [ 4861.858478] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff92354b072800.ost_server_uuid in FULL state after 0 sec [ 4863.146907] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 4876.993521] Lustre: lustre-OST0001: Not available for connect from 192.168.203.18@tcp (stopping) [ 4877.796523] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4877.800444] Lustre: Skipped 1 previous similar message [ 4888.031332] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4888.073829] Lustre: server umount lustre-OST0001 complete [ 4892.128400] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4892.133796] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4893.154590] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4893.156839] LustreError: 314813:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4893.163226] LustreError: 314813:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 4896.209148] Lustre: server umount lustre-OST0000 complete [ 4898.273150] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4903.581132] Lustre: server umount lustre-MDT0000 complete [ 4904.801347] LustreError: 317233:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781032475 with bad export cookie 1675216977267402768 [ 4904.804824] LustreError: 317233:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4904.805322] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4904.921872] Lustre: server umount lustre-MDT0001 complete [ 4908.340127] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing unload_modules_local [ 4909.343979] Key type lgssc unregistered [ 4909.478380] LNet: 318253:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4909.480953] LNetError: 318253:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4909.492289] LNet: Removed LNI 192.168.203.118@tcp [ 4909.795689] Key type .llcrypt unregistered [ 4909.797368] Key type ._llcrypt unregistered [ 4918.933318] Key type ._llcrypt registered [ 4918.934537] Key type .llcrypt registered [ 4918.969631] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 4919.282452] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4919.289979] alg: No test for adler32 (adler32-zlib) [ 4920.142603] Lustre: Lustre: Build Version: 2.17.54_1_g05c2c2a [ 4920.221270] LNet: Added LNI 192.168.203.118@tcp [8/256/0/180] [ 4921.799149] Key type lgssc registered [ 4922.161086] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4924.624257] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4924.632028] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4925.784318] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4927.089481] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 4929.208318] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4931.984383] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4932.142755] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4933.518537] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 4936.878239] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4937.015326] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4938.880719] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 4941.110209] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4942.115937] LustreError: 321279:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4942.123914] LustreError: 321279:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4943.643731] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4943.765727] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4945.687880] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 4947.819727] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 4949.995090] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 4949.998643] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 4954.939487] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 4955.051342] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5030.880239] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5030.885352] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5030.893732] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5031.908452] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5031.909793] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5031.913768] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5031.915719] Lustre: Skipped 1 previous similar message [ 5036.345757] Lustre: server umount lustre-OST0001 complete [ 5037.027129] LustreError: 322812:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5037.031733] LustreError: 322812:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5039.584760] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5039.587124] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5039.592034] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5042.146506] LustreError: 321279:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5042.146963] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5042.147426] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5042.147883] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5042.150941] LustreError: 321279:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5042.153017] Lustre: Skipped 1 previous similar message [ 5047.265347] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5047.266041] LustreError: 321281:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5047.274905] LustreError: 321281:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5052.383188] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5052.388555] LustreError: 321280:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5052.398376] LustreError: 321280:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5052.459725] Lustre: server umount lustre-OST0000 complete [ 5054.944275] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5054.948362] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5057.505434] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5057.510269] Lustre: Skipped 5 previous similar messages [ 5068.255111] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5068.353823] Lustre: server umount lustre-MDT0000 complete [ 5070.129884] LustreError: 319614:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781032641 with bad export cookie 4447686139760224715 [ 5070.134586] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5070.140240] LustreError: 319614:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5070.273971] Lustre: server umount lustre-MDT0001 complete [ 5073.871265] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing unload_modules_local [ 5075.054400] Key type lgssc unregistered [ 5075.177448] LNet: 324370:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5075.180629] LNetError: 324370:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5075.190370] LNet: Removed LNI 192.168.203.118@tcp [ 5075.517639] Key type .llcrypt unregistered [ 5075.518803] Key type ._llcrypt unregistered [ 5095.248293] Key type ._llcrypt registered [ 5095.249543] Key type .llcrypt registered [ 5095.296716] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 5095.627774] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5095.660540] alg: No test for adler32 (adler32-zlib) [ 5096.517080] Lustre: Lustre: Build Version: 2.17.54_1_g05c2c2a [ 5096.610072] LNet: Added LNI 192.168.203.118@tcp [8/256/0/180] [ 5098.191162] Key type lgssc registered [ 5098.776799] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5102.420810] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5102.426574] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5103.619778] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5105.277294] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 5108.899035] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5109.023488] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5110.225215] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 5111.872634] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 5113.978378] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5114.093958] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5116.299517] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 5119.210742] LustreError: 328172:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5119.221892] LustreError: 328172:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5120.800286] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5120.890975] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5123.084911] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 5128.169678] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 5128.169916] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 5132.542534] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5140.671431] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 5147.422388] Lustre: server umount lustre-MDT0000 complete [ 5148.585535] LustreError: 327386:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781032719 with bad export cookie 10349841316640094415 [ 5148.591884] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5148.640485] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5148.641432] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5148.641714] LustreError: 326612:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5148.643924] Lustre: Skipped 2 previous similar messages [ 5148.651579] Lustre: Skipped 1 previous similar message [ 5150.694214] Lustre: server umount lustre-MDT0001 complete [ 5152.045620] Lustre: server umount lustre-OST0000 complete [ 5153.577507] Lustre: server umount lustre-OST0001 complete [ 5160.026469] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 5164.410938] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5164.607913] LustreError: 331982:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5164.617101] LustreError: 331982:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5164.635932] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5166.086451] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 5169.424368] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5170.922608] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 5172.724987] Lustre: 333187:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5175.709104] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5175.905450] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5175.910361] Lustre: Skipped 1 previous similar message [ 5178.273921] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 5178.987174] LustreError: 333540:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5182.484625] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5184.852532] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 5189.099072] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 5189.101167] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 5194.063501] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5195.433785] Lustre: 335116:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5199.329308] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5199.332571] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5199.333395] Lustre: Skipped 3 previous similar messages [ 5199.335671] Lustre: Skipped 1 previous similar message [ 5202.392931] Lustre: server umount lustre-MDT0000 complete [ 5203.550986] LustreError: 335118:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781032774 with bad export cookie 10349841316640096074 [ 5203.554140] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5203.554229] LustreError: 335118:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 5203.659609] Lustre: server umount lustre-MDT0001 complete [ 5215.324266] Lustre: server umount lustre-OST0000 complete [ 5227.503840] Lustre: server umount lustre-OST0001 complete [ 5229.269077] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 5230.076172] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 5230.927304] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 15:20:00 (1781032800) [ 5240.657720] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 5244.453046] loop0: detected capacity change from 0 to 1099511627776 [ 5244.633402] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 5244.635679] *** VALIDATE xfs *** [ 5244.639846] XFS (loop0): Mounting V5 Filesystem [ 5244.649813] XFS (loop0): Ending clean mount [ 5245.264935] loop1: detected capacity change from 0 to 18691697672192 [ 5268.368715] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_hostid [ 5272.484657] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 5289.355489] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5289.449315] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5289.464375] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5289.512778] Lustre: lustre-MDT0000: new disk, initializing [ 5289.545631] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5289.549106] Lustre: Skipped 1 previous similar message [ 5289.584795] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5290.878170] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 5295.985312] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5296.036423] Lustre: 340177:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5296.070908] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5296.072776] Lustre: Skipped 1 previous similar message [ 5296.114223] Lustre: lustre-MDT0001: new disk, initializing [ 5296.169256] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5296.173895] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5297.474225] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 5301.386985] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5303.279252] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5305.743943] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5305.846123] Lustre: lustre-OST0000: new disk, initializing [ 5305.848948] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5305.852777] Lustre: 342017:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5307.516975] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5307.520212] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5307.556715] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5307.625919] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 5312.064805] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5313.896103] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5313.964402] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5315.320938] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 5315.381117] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5318.117595] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5318.118476] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5318.122645] LustreError: Skipped 1 previous similar message [ 5318.132041] Lustre: Skipped 1 previous similar message [ 5318.136109] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5318.143413] Lustre: Skipped 3 previous similar messages [ 5323.234783] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5323.237942] Lustre: Skipped 1 previous similar message [ 5328.355598] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5328.359340] Lustre: Skipped 1 previous similar message [ 5330.399137] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5330.458845] Lustre: server umount lustre-OST0000 complete [ 5333.474654] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5333.481486] Lustre: Skipped 1 previous similar message [ 5338.280243] Lustre: server umount lustre-MDT0000 complete [ 5338.592257] LustreError: 340188:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5338.599590] LustreError: 340188:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5339.648241] LustreError: 340170:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781032910 with bad export cookie 10349841316640097607 [ 5339.651954] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5339.652403] LustreError: 340170:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5339.769522] Lustre: server umount lustre-MDT0001 complete [ 5342.911524] XFS (loop0): Unmounting Filesystem [ 5344.766486] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 15:21:54 (1781032914) [ 5347.335646] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5347.482802] LustreError: 344472:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5347.505673] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5347.507388] Lustre: Skipped 2 previous similar messages [ 5348.731192] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 5351.884202] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5353.331361] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 5355.449061] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5356.741293] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5359.812328] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5362.318069] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 5365.410180] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5382.625112] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5382.628518] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5382.633899] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5382.635785] Lustre: Skipped 2 previous similar messages [ 5383.649928] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5385.491758] Lustre: server umount lustre-OST0000 complete [ 5387.744061] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5399.009294] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5399.013721] Lustre: Skipped 8 previous similar messages [ 5401.055378] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5401.144783] Lustre: server umount lustre-MDT0000 complete [ 5402.500546] LustreError: 346888:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781032973 with bad export cookie 10349841316640098489 [ 5402.505862] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5402.507105] LustreError: 346888:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5402.629316] Lustre: server umount lustre-MDT0001 complete [ 5406.859701] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing unload_modules_local [ 5407.843805] Key type lgssc unregistered [ 5407.971515] LNet: 347910:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5407.974649] LNetError: 347910:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5407.993359] LNet: Removed LNI 192.168.203.118@tcp [ 5408.356193] Key type .llcrypt unregistered [ 5408.357460] Key type ._llcrypt unregistered [ 5419.651699] Key type ._llcrypt registered [ 5419.652642] Key type .llcrypt registered [ 5419.689183] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 5419.977310] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5419.985358] alg: No test for adler32 (adler32-zlib) [ 5420.853553] Lustre: Lustre: Build Version: 2.17.54_1_g05c2c2a [ 5420.942051] LNet: Added LNI 192.168.203.118@tcp [8/256/0/180] [ 5422.527161] Key type lgssc registered [ 5423.107341] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5426.280359] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5426.287367] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5427.455388] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5428.680148] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 5432.397191] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5432.557708] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5433.758427] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 5436.050069] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5437.433941] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5440.088332] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5440.230860] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5442.241729] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 5444.733246] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5458.399977] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 5458.403098] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5458.408648] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5460.154395] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.18@tcp (stopping) [ 5463.011567] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5463.011791] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5464.156590] Lustre: server umount lustre-MDT0001 complete [ 5465.265517] LustreError: 349431:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.18@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5465.273825] LustreError: 349431:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5467.989560] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5468.019569] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5468.021212] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5468.034844] Lustre: 349423:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5468.150287] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5469.542823] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 5473.262359] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 5475.307216] LustreError: 353006:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5475.314944] LustreError: 353006:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5475.322124] LustreError: 349427:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5475.355080] LustreError: 353009:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.118@tcp: cfg command failed: rc = -17 [ 5475.361599] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 5475.361599] [ 5772.474107] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.18@tcp (stopping) [ 5772.476760] Lustre: Skipped 1 previous similar message [ 5775.329062] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5775.330811] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 5775.334123] Lustre: Skipped 1 previous similar message [ 5777.586727] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.18@tcp (stopping) [ 5777.591755] Lustre: Skipped 2 previous similar messages [ 5778.419155] Lustre: server umount lustre-MDT0001 complete [ 5780.448731] LustreError: 350792:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5780.454512] LustreError: 350792:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5782.234701] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5782.270855] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5782.272669] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5782.289471] Lustre: 349425:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5782.407468] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5783.684988] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 5787.623678] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 5791.210001] LustreError: 364959:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5791.214764] LustreError: 364959:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5791.219992] LustreError: 349427:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5791.223909] LustreError: 349427:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 5791.257501] LustreError: 364962:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [ 5791.264673] LustreError: 364962:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [ 5791.270722] LustreError: 364962:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.118@tcp: cfg command failed: rc = -17 [ 5791.277152] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.203.118@tcp [ 5791.277152] [ 6089.702228] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6089.704070] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 6089.712597] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6089.719500] Lustre: Skipped 1 previous similar message [ 6092.972869] Lustre: server umount lustre-MDT0001 complete [ 6094.816382] LustreError: 350792:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6094.820414] LustreError: 350792:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6097.262261] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6097.314976] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6097.316778] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6097.335485] Lustre: 349425:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6097.489057] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 6099.183999] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 6102.508332] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6107.637430] LustreError: 376760:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6107.643819] LustreError: 376760:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6107.650709] LustreError: 349427:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6107.655850] LustreError: 349427:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 6107.685609] LustreError: 376763:0:(lod_lov.c:212:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 6107.697220] LustreError: 376763:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.118@tcp: cfg command failed: rc = -114 [ 6107.703576] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 6107.703576] [ 6419.939767] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6419.941078] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6419.941346] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6419.941355] Lustre: Skipped 1 previous similar message [ 6419.946625] Lustre: Skipped 2 previous similar messages [ 6421.559266] Lustre: server umount lustre-MDT0000 complete [ 6423.303196] LustreError: 349416:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781033994 with bad export cookie 12564290238403955576 [ 6423.310288] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6423.310747] LustreError: 349416:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6425.056949] LustreError: 369283:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6425.070412] LustreError: 369283:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6429.526228] Lustre: server umount lustre-MDT0001 complete [ 6431.435755] Lustre: server umount lustre-OST0000 complete [ 6434.541165] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_hostid [ 6438.696652] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 6455.552875] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 15:40:25 (1781034025) [ 6458.532961] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6458.657216] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6458.674434] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6458.726139] Lustre: lustre-MDT0000: new disk, initializing [ 6458.760487] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6458.774230] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6460.548565] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 6466.521134] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6466.591418] Lustre: 390290:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6466.628263] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6466.632380] Lustre: Skipped 1 previous similar message [ 6466.682265] Lustre: lustre-MDT0001: new disk, initializing [ 6466.718837] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6466.732547] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6466.735517] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6468.406556] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 6472.893637] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6474.531726] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6477.527427] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6477.630393] Lustre: lustre-OST0000: new disk, initializing [ 6477.632409] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6477.635440] Lustre: 392134:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6477.664365] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6479.695211] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_default_debug -1 all [ 6483.468420] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6483.475099] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6483.526116] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6483.718055] Lustre: DEBUG MARKER: oleg318-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6487.520324] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6487.525161] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6487.532609] Lustre: Skipped 2 previous similar messages [ 6487.535952] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6487.539925] Lustre: Skipped 4 previous similar messages [ 6492.449835] Lustre: server umount lustre-MDT0000 complete [ 6493.642366] LustreError: 390284:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781034064 with bad export cookie 12564290238403957515 [ 6493.646761] LustreError: MGC192.168.203.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6493.648857] LustreError: 390284:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6493.665081] LustreError: 392730:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6493.670972] LustreError: 392730:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6493.763403] Lustre: server umount lustre-MDT0001 complete [ 6501.359379] Lustre: server umount lustre-OST0000 complete [ 6514.607052] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing set_hostid [ 6518.762565] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing load_modules_local [ 6533.600325] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 6249 sec ========== 15:41:43 (1781034103) [ 6534.465974] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 15:41:44 (1781034104) === [ 6536.017366] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 15:41:45 (1781034105) === [ 6555.077400] Lustre: DEBUG MARKER: oleg318-server.virtnet: executing unload_modules_local [ 6556.685206] Key type lgssc unregistered [ 6556.868742] LNet: 398419:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6556.873659] LNetError: 398419:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6556.886708] LNet: Removed LNI 192.168.203.118@tcp [ 6557.330312] Key type .llcrypt unregistered [ 6557.332604] Key type ._llcrypt unregistered