[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 478940462 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.003051] x2apic enabled [ 0.004006] Switched APIC routing to physical x2apic. [ 0.005012] kvm-guest: setup PV IPIs [ 0.007621] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008023] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009012] pid_max: default: 32768 minimum: 301 [ 0.010148] LSM: Security Framework initializing [ 0.011042] Yama: becoming mindful. [ 0.013030] SELinux: Initializing. [ 0.014050] *** VALIDATE selinux *** [ 0.022052] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026697] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.027153] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028119] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029163] *** VALIDATE tmpfs *** [ 0.031204] *** VALIDATE proc *** [ 0.032248] *** VALIDATE cgroup *** [ 0.033008] *** VALIDATE cgroup2 *** [ 0.035247] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.036158] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.037008] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.038027] Spectre V2 : User space: Vulnerable [ 0.039005] Speculative Store Bypass: Vulnerable [ 0.042053] debug: unmapping init [mem 0xffffffffa6659000-0xffffffffa6660fff] [ 0.045206] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.046780] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.047026] ... version: 2 [ 0.048013] ... bit width: 48 [ 0.049023] ... generic registers: 4 [ 0.050014] ... value mask: 0000ffffffffffff [ 0.051017] ... max period: 00007fffffffffff [ 0.052018] ... fixed-purpose events: 3 [ 0.053014] ... event mask: 000000070000000f [ 0.055274] rcu: Hierarchical SRCU implementation. [ 0.057595] smp: Bringing up secondary CPUs ... [ 0.058699] x86: Booting SMP configuration: [ 0.059048] .... node #0, CPUs: #1 #2 #3 [ 0.069098] smp: Brought up 1 node, 4 CPUs [ 0.071016] smpboot: Max logical packages: 1 [ 0.072016] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.148025] node 0 deferred pages initialised in 75ms [ 0.153109] devtmpfs: initialized [ 0.154216] x86/mm: Memory block size: 128MB [ 0.156630] gcov: version magic: 0x41383552 [ 0.159301] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.163097] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.168300] pinctrl core: initialized pinctrl subsystem [ 0.170209] [ 0.171008] ************************************************************* [ 0.174014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.176016] ** ** [ 0.179018] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.181008] ** ** [ 0.182007] ** This means that this kernel is built to expose internal ** [ 0.184008] ** IOMMU data structures, which may compromise security on ** [ 0.186010] ** your system. ** [ 0.188014] ** ** [ 0.190014] ** If you see this message and you are not debugging the ** [ 0.192017] ** kernel, report this immediately to your vendor! ** [ 0.194013] ** ** [ 0.196014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.199013] ************************************************************* [ 0.201994] NET: Registered protocol family 16 [ 0.203594] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.207138] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.210062] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.212782] cpuidle: using governor menu [ 0.214000] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.219185] PCI: Using configuration type 1 for base access [ 0.222150] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.229105] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.231062] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.235511] cryptd: max_cpu_qlen set to 1000 [ 0.236402] ACPI: Added _OSI(Module Device) [ 0.238014] ACPI: Added _OSI(Processor Device) [ 0.240025] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.242011] ACPI: Added _OSI(Processor Aggregator Device) [ 0.247374] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.255012] ACPI: Interpreter enabled [ 0.257056] ACPI: PM: (supports S0 S3 S4 S5) [ 0.260012] ACPI: Using IOAPIC for interrupt routing [ 0.263420] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.267289] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.285641] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.290068] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.299025] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.307168] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.314000] acpiphp: Slot [2] registered [ 0.315196] acpiphp: Slot [5] registered [ 0.318164] acpiphp: Slot [6] registered [ 0.320209] acpiphp: Slot [7] registered [ 0.321120] acpiphp: Slot [8] registered [ 0.322132] acpiphp: Slot [9] registered [ 0.324232] acpiphp: Slot [10] registered [ 0.326156] acpiphp: Slot [3] registered [ 0.328102] acpiphp: Slot [4] registered [ 0.330127] acpiphp: Slot [11] registered [ 0.332151] acpiphp: Slot [12] registered [ 0.333161] acpiphp: Slot [13] registered [ 0.334000] acpiphp: Slot [14] registered [ 0.334000] acpiphp: Slot [15] registered [ 0.334000] acpiphp: Slot [16] registered [ 0.334000] acpiphp: Slot [17] registered [ 0.334000] acpiphp: Slot [18] registered [ 0.334000] acpiphp: Slot [19] registered [ 0.334000] acpiphp: Slot [20] registered [ 0.336148] acpiphp: Slot [21] registered [ 0.337000] acpiphp: Slot [22] registered [ 0.340101] acpiphp: Slot [23] registered [ 0.341081] acpiphp: Slot [24] registered [ 0.342081] acpiphp: Slot [25] registered [ 0.344079] acpiphp: Slot [26] registered [ 0.345093] acpiphp: Slot [27] registered [ 0.346087] acpiphp: Slot [28] registered [ 0.352166] acpiphp: Slot [29] registered [ 0.355201] acpiphp: Slot [30] registered [ 0.358115] acpiphp: Slot [31] registered [ 0.360103] PCI host bridge to bus 0000:00 [ 0.362024] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.365026] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.368025] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.371023] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.374022] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.376028] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.378158] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.381069] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.385301] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.394015] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.400654] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.404022] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.406016] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.409020] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.412657] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.415889] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.419051] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.421940] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.426018] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.446013] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.451021] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.455877] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.461042] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.468026] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.487019] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.503622] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.520019] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.529023] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.544015] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.559000] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.569021] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.571000] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.598027] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.615952] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.623496] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.637019] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.655021] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.671431] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.680308] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.688016] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.714015] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.727113] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.736017] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.744019] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.767022] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.785281] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.788390] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.792102] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.795432] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.800288] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.809087] iommu: Default domain type: Passthrough [ 0.810499] SCSI subsystem initialized [ 0.812135] ACPI: bus type USB registered [ 0.813086] usbcore: registered new interface driver usbfs [ 0.815068] usbcore: registered new interface driver hub [ 0.817093] usbcore: registered new device driver usb [ 0.819147] pps_core: LinuxPPS API ver. 1 registered [ 0.821010] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.824056] PTP clock support registered [ 0.827009] EDAC MC: Ver: 3.0.0 [ 0.828934] PCI: Using ACPI for IRQ routing [ 0.830952] NetLabel: Initializing [ 0.832011] NetLabel: domain hash size = 128 [ 0.834010] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.836085] NetLabel: unlabeled traffic allowed by default [ 0.839003] vgaarb: loaded [ 0.840697] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.842010] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.848288] clocksource: Switched to clocksource kvm-clock [ 0.961389] VFS: Disk quotas dquot_6.6.0 [ 0.963024] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.965575] *** VALIDATE ramfs *** [ 0.966756] *** VALIDATE hugetlbfs *** [ 0.968471] pnp: PnP ACPI init [ 0.971100] pnp: PnP ACPI: found 6 devices [ 0.987950] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.991537] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.994185] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.996681] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.999291] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.001701] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.004549] NET: Registered protocol family 2 [ 1.007027] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.012194] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.016117] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.021574] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.025074] TCP: Hash tables configured (established 65536 bind 65536) [ 1.028152] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.031559] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.034558] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.037040] NET: Registered protocol family 1 [ 1.039115] RPC: Registered named UNIX socket transport module. [ 1.041081] RPC: Registered udp transport module. [ 1.043101] RPC: Registered tcp transport module. [ 1.044955] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.047833] NET: Registered protocol family 44 [ 1.049857] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.052450] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.054872] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.057672] PCI: CLS 0 bytes, default 64 [ 1.060223] Unpacking initramfs... [ 2.875824] debug: unmapping init [mem 0xffff9bc2bcc54000-0xffff9bc2bffbffff] [ 2.881141] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.883935] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.898647] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.618223] Initialise system trusted keyrings [ 3.620513] Key type blacklist registered [ 3.622708] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.636380] zbud: loaded [ 3.640560] *** VALIDATE nfs *** [ 3.642274] *** VALIDATE nfs4 *** [ 3.644348] pstore: using deflate compression [ 3.649022] Platform Keyring initialized [ 3.792428] NET: Registered protocol family 38 [ 3.794952] Key type asymmetric registered [ 3.797440] Asymmetric key parser 'x509' registered [ 3.800305] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.804636] io scheduler mq-deadline registered [ 3.807681] io scheduler kyber registered [ 3.810494] io scheduler bfq registered [ 3.812551] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.818145] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.825195] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.832474] ACPI: Power Button [PWRF] [ 3.840390] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.850778] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.870501] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.888293] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.967100] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 4.002884] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 4.032962] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 4.043232] Non-volatile memory driver v1.3 [ 4.045055] Linux agpgart interface v0.103 [ 4.077651] virtio_blk virtio1: [vda] 146168 512-byte logical blocks (74.8 MB/71.4 MiB) [ 4.080680] vda: detected capacity change from 0 to 74838016 [ 4.105301] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 4.109579] vdb: detected capacity change from 0 to 1073741824 [ 4.134592] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 4.138141] vdc: detected capacity change from 0 to 2621440000 [ 4.156332] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 4.159480] vdd: detected capacity change from 0 to 2621440000 [ 4.175493] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 4.178808] vde: detected capacity change from 0 to 4294967296 [ 4.196431] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 4.199825] vdf: detected capacity change from 0 to 4294967296 [ 4.208514] libphy: Fixed MDIO Bus: probed [ 4.213754] usbcore: registered new interface driver usbserial_generic [ 4.216652] usbserial: USB Serial support registered for generic [ 4.219640] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 4.224966] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 4.227140] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 4.229736] mousedev: PS/2 mouse device common for all mice [ 4.234650] rtc_cmos 00:05: RTC can wake from S4 [ 4.238216] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 4.243070] rtc_cmos 00:05: registered as rtc0 [ 4.253083] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 4.266885] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 4.268982] intel_pstate: CPU model not supported [ 4.296669] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 4.298227] hid: raw HID events driver (C) Jiri Kosina [ 4.311816] usbcore: registered new interface driver usbhid [ 4.318345] usbhid: USB HID core driver [ 4.323522] drop_monitor: Initializing network drop monitor service [ 4.333498] Initializing XFRM netlink socket [ 4.346983] NET: Registered protocol family 10 [ 4.371568] Segment Routing with IPv6 [ 4.377816] NET: Registered protocol family 17 [ 4.380845] mpls_gso: MPLS GSO support [ 4.393573] RAS: Correctable Errors collector initialized. [ 4.399139] AVX version of gcm_enc/dec engaged. [ 4.403044] AES CTR mode by8 optimization enabled [ 4.577563] sched_clock: Marking stable (4577497097, 0)->(5527090244, -949593147) [ 4.586441] registered taskstats version 1 [ 4.588713] Loading compiled-in X.509 certificates [ 4.590738] zswap: loaded using pool lzo/zbud [ 4.619590] Key type big_key registered [ 4.632156] Key type encrypted registered [ 4.633759] ima: No TPM chip found, activating TPM-bypass! [ 4.636245] ima: Allocated hash algorithm: sha1 [ 4.638241] ima: No architecture policies found [ 4.640282] evm: Initialising EVM extended attributes: [ 4.642710] evm: security.selinux [ 4.643890] evm: security.ima [ 4.645120] evm: security.capability [ 4.646423] evm: HMAC attrs: 0x1 [ 4.648673] rtc_cmos 00:05: setting system clock to 2026-09-01 13:26:38 UTC (1788269198) [ 4.658392] debug: unmapping init [mem 0xffffffffa7603000-0xffffffffa77fffff] [ 4.661851] debug: unmapping init [mem 0xffffffffa6382000-0xffffffffa6658fff] [ 4.671179] Write protecting the kernel read-only data: 28672k [ 4.682421] debug: unmapping init [mem 0xffffffffa4a03000-0xffffffffa4bfffff] [ 4.972023] debug: unmapping init [mem 0xffffffffa5314000-0xffffffffa53fffff] [ 5.008099] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 5.016496] systemd[1]: Detected virtualization kvm. [ 5.018640] systemd[1]: Detected architecture x86-64. [ 5.020784] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 5.049928] systemd[1]: No hostname configured. [ 5.051590] systemd[1]: Set hostname to . [ 5.053974] random: systemd: uninitialized urandom read (16 bytes read) [ 5.056599] systemd[1]: Initializing machine ID from random generator. [ 5.178149] random: ln: uninitialized urandom read (6 bytes read) [ 5.576892] random: systemd: uninitialized urandom read (16 bytes read) [ 5.581120] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 5.603428] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ 5.620377] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Swap. [ OK ] Listening on Journal Socket. Starting Apply Kernel Variables... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Listening on udev Kernel Socket. [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Timers. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... Starting Create list of required st…ce nodes for the current kernel... Starting Setup Virtual Console... [ OK ] Reached target Sockets. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Journal Service. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 7.674746] device-mapper: uevent: version 1.0.3 [ 7.677977] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... [ 8.398936] random: fast init done Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 9.289265] virtio_net virtio0 ens2: renamed from eth0 [ 9.697463] scsi host0: ata_piix [ 9.814164] scsi host1: ata_piix [ 9.817017] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 9.820127] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.853804] hrtimer: interrupt took 4991673 ns [ 14.191716] random: crng init done [ 14.194626] random: 7 urandom warning(s) missed due to ratelimiting [ 16.979238] dracut-initqueue[593]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 18.526992] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Timers. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped target Swap. Stopping udev Kernel Device Manager... [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Sockets. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 21.353454] printk: systemd: 23 output lines suppressed due to ratelimiting [ 22.035698] SELinux: Disabled at runtime. [ 22.190716] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 22.206849] systemd[1]: Detected virtualization kvm. [ 22.208727] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 23.842224] systemd[1]: initrd-switch-root.service: Succeeded. [ 23.854414] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 23.891934] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 23.900876] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 23.905651] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 23.920523] systemd[1]: Starting Journal Service... Starting Journal Service... [ 23.944249] systemd[1]: Created slice User and Session Slice. [ OK ] Created slice User and Session Slice. [ OK ] Stopped target Switch Root. [ OK ] Reached target Slices. [ OK ] Listening on Process Core Dump Socket. Activating swap /dev/disk/by-label/SWAP... [ OK ] Created slice system-sshd\x2dkeygen.slice. Mounting Huge Pages File System... [ OK ] Created slice system-serial\x[ 24.129924] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS 2dgetty.slice. Starting Create list of required st…ce nodes for the current kernel... Starting Remount Root and Kernel File Systems... Starting Apply Kernel Variables... [ OK ] Stopped target Initrd Root File System. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on initctl Compatibility Named Pipe. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Stopped target Initrd File Systems. Mounting POSIX Message Queue File System... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Created slice system-getty.slice. Mounting Kernel Debug File System... [ OK ] Reached target rpc_pipefs.target. [ OK ] Started Journal Service. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Huge Pages File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Kernel Debug File System. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... Starting udev Kernel Device Manager... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 25.641174] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 26.713137] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 26.915202] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 27.706406] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 27.885831] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (7s / no limit) [** ] A start job is running for Configur…-only root support (8s / no limit) [*** ] A start job is running for Configur…-only root support (8s / no limit) [ *** ] A start job is running for Configur…-only root support (9s / no limit) [ *** ] A start job is running for Configur…-only root support (9s / no limit)[ 33.650182] Key type dns_resolver registered [ ***] A start job is running for Configur…only root support (10s / no limit)[ 34.172144] NFS: Registering the id_resolver key type [ 34.176133] Key type id_resolver registered [ 34.179911] Key type id_legacy registered [ **] A start job is running for Configur…only root support (10s / no limit) [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started dnf makecache --timer. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. [ OK ] Started D-Bus System Message Bus. Starting Login Service... Starting Network Manager... [ OK ] Reached target sshd-keygen.target. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. Starting Restore /run/initramfs on shutdown... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Started Login Service. Starting Hostname Service... [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg307-server login: [ 81.110693] libcfs: loading out-of-tree module taints kernel. [ 81.133126] Key type ._llcrypt registered [ 81.135665] Key type .llcrypt registered [ 81.194312] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [ 91.501723] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 92.304462] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 92.315308] alg: No test for adler32 (adler32-zlib) [ 93.413678] Lustre: Lustre: Build Version: 2.17.57_82_g90dbd01 [ 93.827940] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 95.479155] Key type lgssc registered [ 96.343438] Lustre: Echo OBD driver; http://www.lustre.org/ [ 105.689391] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 128.725633] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 136.101697] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 136.126819] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 137.291162] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 137.316687] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 137.381174] Lustre: lustre-MDT0000: new disk, initializing [ 137.427313] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 137.445250] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 139.847251] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 147.457403] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 147.519546] Lustre: 6487:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 147.543495] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 147.549847] Lustre: Skipped 1 previous similar message [ 147.610044] Lustre: lustre-MDT0001: new disk, initializing [ 147.648232] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 147.663795] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 147.671208] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 150.100878] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 153.139951] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 158.439772] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 158.570953] Lustre: lustre-OST0000: new disk, initializing [ 158.573993] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 158.578681] Lustre: 8419:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 158.613524] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 160.961080] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 160.966560] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 161.012661] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 161.807197] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 171.345372] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 171.452583] Lustre: lustre-OST0001: new disk, initializing [ 171.455580] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 171.459308] Lustre: 9492:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 171.508706] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 175.186761] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 177.676094] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 177.681722] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 177.729258] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 182.633658] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 187.432480] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 190.764559] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing check_logdir /tmp/testlogs/ [ 193.697310] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing yml_node [ 196.262508] Lustre: DEBUG MARKER: Client: 2.17.57.82 [ 197.698365] Lustre: DEBUG MARKER: MDS: 2.17.57.82 [ 199.198270] Lustre: DEBUG MARKER: OSS: 2.17.57.82 [ 200.163106] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Tue Sep 1 09:29:53 EDT 2026 [ 209.820149] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 210.832412] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 223.712553] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 223.713300] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 223.722092] Lustre: Skipped 3 previous similar messages [ 228.833478] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 228.837495] Lustre: Skipped 6 previous similar messages [ 229.946160] Lustre: server umount lustre-MDT0000 complete [ 231.807781] LustreError: 7433:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788269425 with bad export cookie 11406558740267414422 [ 231.809399] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 231.816718] LustreError: 7433:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 233.955146] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 233.962240] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 233.963278] Lustre: Skipped 1 previous similar message [ 233.974658] Lustre: Skipped 1 previous similar message [ 236.960360] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 236.964978] Lustre: Skipped 1 previous similar message [ 242.084811] LustreError: 8672:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 242.085072] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 242.100386] LustreError: 8672:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 242.113188] Lustre: Skipped 1 previous similar message [ 246.239240] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 246.388109] Lustre: server umount lustre-MDT0001 complete [ 249.181876] Lustre: server umount lustre-OST0000 complete [ 251.727785] Lustre: server umount lustre-OST0001 complete [ 255.877394] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [ 261.820492] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 283.181707] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 283.315083] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 283.335859] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 283.380484] Lustre: lustre-MDT0000: new disk, initializing [ 283.426849] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 283.450591] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 285.722377] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 291.869580] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 291.945545] Lustre: 15035:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 291.989376] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 291.993234] Lustre: Skipped 1 previous similar message [ 292.045637] Lustre: lustre-MDT0001: new disk, initializing [ 292.096436] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 292.122942] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 292.131662] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 294.397881] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 299.409243] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 302.144254] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 305.428278] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 305.578943] Lustre: 17000:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 307.600183] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 307.662265] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 308.497932] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 314.042244] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 316.444438] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 316.549597] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 318.659641] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 318.766914] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 323.043906] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 323.048052] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 323.051679] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 323.057909] Lustre: Skipped 1 previous similar message [ 334.303387] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 334.473312] Lustre: server umount lustre-OST0000 complete [ 338.401855] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 338.418433] Lustre: Skipped 1 previous similar message [ 338.421644] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 338.426918] LustreError: Skipped 1 previous similar message [ 343.319477] Lustre: server umount lustre-MDT0000 complete [ 343.520275] LustreError: 15957:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 343.528889] LustreError: 15957:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 345.563536] LustreError: 15025:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788269539 with bad export cookie 11406558740267415955 [ 345.575660] LustreError: 15025:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 345.577356] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 345.740141] Lustre: server umount lustre-MDT0001 complete [ 348.597746] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 09:32:21 (1788269541) [ 352.020726] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 352.320300] LustreError: 19268:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 352.361826] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 352.365180] Lustre: Skipped 1 previous similar message [ 354.541548] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 357.860589] LustreError: 19269:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 359.567278] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 362.745886] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 367.151485] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 370.185604] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 374.547835] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 378.032522] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 382.544147] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 388.781285] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 395.234178] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 395.238855] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 395.245259] Lustre: Skipped 1 previous similar message [ 395.255772] Lustre: Skipped 9 previous similar messages [ 399.651100] Lustre: server umount lustre-MDT0000 complete [ 400.356897] LustreError: 19262:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 400.371592] LustreError: 19262:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 401.957420] LustreError: 19248:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788269595 with bad export cookie 11406558740267416837 [ 401.957873] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 401.966227] LustreError: 19248:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 402.196326] Lustre: server umount lustre-MDT0001 complete [ 415.180539] Lustre: server umount lustre-OST0000 complete [ 429.037292] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 435.450470] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 435.712320] LustreError: 23647:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 435.755955] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 435.759210] Lustre: Skipped 2 previous similar messages [ 438.253327] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 443.576049] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 446.441703] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 448.296856] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 452.349912] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 455.975865] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 460.672918] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 460.814207] Lustre: lustre-OST0001: new disk, initializing [ 460.817511] Lustre: Skipped 1 previous similar message [ 460.820423] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 460.823587] Lustre: Skipped 1 previous similar message [ 460.827918] Lustre: 25963:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 462.398417] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 462.406291] Lustre: Skipped 1 previous similar message [ 462.410604] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 462.474277] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 464.510497] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 470.620982] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 472.337651] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 479.713742] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 479.722241] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 479.730888] Lustre: Skipped 1 previous similar message [ 479.736324] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 485.519796] Lustre: server umount lustre-MDT0000 complete [ 487.562977] LustreError: 23629:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788269681 with bad export cookie 11406558740267417684 [ 487.564238] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 487.573062] LustreError: 23629:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 487.762205] Lustre: server umount lustre-MDT0001 complete [ 499.239463] Lustre: server umount lustre-OST0000 complete [ 511.576227] Lustre: server umount lustre-OST0001 complete [ 520.482358] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 525.680120] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 525.873549] LustreError: 28977:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 525.885990] LustreError: 28977:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 525.919167] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 525.923366] Lustre: Skipped 3 previous similar messages [ 527.900520] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 532.108745] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 534.240903] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 535.726503] Lustre: 30123:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 539.181957] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 543.043291] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 547.899584] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 551.040031] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 555.166577] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 556.902562] Lustre: 32002:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 561.634670] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 561.639517] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 561.645971] Lustre: Skipped 5 previous similar messages [ 561.653051] Lustre: Skipped 6 previous similar messages [ 565.453893] Lustre: server umount lustre-MDT0000 complete [ 566.752631] LustreError: 28978:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 566.762881] LustreError: 28978:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 567.553741] LustreError: 28959:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788269761 with bad export cookie 11406558740267418979 [ 567.555725] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 567.565974] LustreError: 28959:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 567.754835] Lustre: server umount lustre-MDT0001 complete [ 580.121898] Lustre: server umount lustre-OST0000 complete [ 591.368370] Lustre: server umount lustre-OST0001 complete [ 596.402247] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 09:36:29 (1788269789) [ 612.983989] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 618.031637] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 620.695712] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 625.559219] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 628.435483] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 630.213818] Lustre: 35948:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 634.167738] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 635.501776] LustreError: 36429:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 635.525551] LustreError: 36429:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 637.790118] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 642.977790] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 646.507561] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 650.714436] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 652.538901] Lustre: 37831:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 656.352800] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 656.356610] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 656.362921] Lustre: Skipped 1 previous similar message [ 661.492484] Lustre: server umount lustre-MDT0000 complete [ 663.373353] LustreError: 37833:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788269857 with bad export cookie 11406558740267420085 [ 663.374145] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 663.378636] LustreError: 37833:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 663.541271] Lustre: server umount lustre-MDT0001 complete [ 675.862589] Lustre: server umount lustre-OST0000 complete [ 687.418562] Lustre: server umount lustre-OST0001 complete [ 692.850639] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 09:38:06 (1788269886) [ 701.867187] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 707.064101] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 707.297028] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 707.299883] Lustre: Skipped 7 previous similar messages [ 709.514988] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 714.012164] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 716.506931] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 718.066512] Lustre: 41119:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 721.863528] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 725.371757] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 729.940170] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 732.861579] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 737.506580] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 739.966385] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 739.971365] Lustre: Skipped 1 previous similar message [ 743.396274] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 743.398105] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 743.404397] Lustre: Skipped 6 previous similar messages [ 743.407326] Lustre: Skipped 7 previous similar messages [ 747.649568] Lustre: server umount lustre-MDT0000 complete [ 749.516882] LustreError: 39949:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788269943 with bad export cookie 11406558740267421184 [ 749.521904] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 749.526926] LustreError: 39949:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 755.740119] Lustre: server umount lustre-MDT0001 complete [ 760.111333] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 775.137891] LustreError: 3622:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff9bc203464380 x1875136256209024/t0(0) o250->MGC192.168.203.107@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 775.281203] LustreError: 41481:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 775.290515] LustreError: 41481:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 14 previous similar messages [ 777.385655] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 782.181176] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 784.490710] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 784.784340] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 788.953362] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 789.736343] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 792.562939] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 793.502271] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 797.665139] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 802.256487] Lustre: server umount lustre-MDT0000 complete [ 804.482820] LustreError: 39949:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788269998 with bad export cookie 11406558740267422311 [ 804.483227] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 804.494086] LustreError: 39949:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 804.680139] Lustre: server umount lustre-MDT0001 complete [ 816.984854] Lustre: server umount lustre-OST0000 complete [ 829.245383] Lustre: server umount lustre-OST0001 complete [ 833.747906] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 09:40:27 (1788270027) [ 841.662863] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 846.715688] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 849.116840] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 853.467548] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 855.632272] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 856.997072] Lustre: 48743:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 857.003152] Lustre: 48743:0:(mgs_llog.c:1450:mgs_modify_param()) Skipped 1 previous similar message [ 860.071385] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 862.998222] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 867.134983] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 869.921710] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 879.430463] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 881.622552] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 886.755338] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 886.767394] Lustre: Skipped 9 previous similar messages [ 892.467445] Lustre: server umount lustre-MDT0000 complete [ 894.270333] LustreError: 47575:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788270088 with bad export cookie 11406558740267423102 [ 894.274787] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 894.276041] LustreError: 47575:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 894.433870] Lustre: server umount lustre-MDT0001 complete [ 907.204406] Lustre: server umount lustre-OST0000 complete [ 918.961973] Lustre: server umount lustre-OST0001 complete [ 922.929070] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 09:41:56 (1788270116) [ 923.881052] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 924.613538] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 09:41:58 (1788270118) [ 929.026576] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [ 932.575391] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 942.167694] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 942.288132] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 942.332915] Lustre: lustre-MDT0000: new disk, initializing [ 942.373870] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 944.157520] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 948.444033] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 951.290436] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 951.399899] Lustre: 54815:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 952.778499] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 952.795109] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 953.875970] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 958.278800] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 959.103985] Lustre: DEBUG MARKER: create test files [ 973.281478] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 977.111871] Lustre: server umount lustre-OST0000 complete [ 978.962799] Lustre: server umount lustre-MDT0000 complete [ 983.329293] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 984.571444] Key type lgssc unregistered [ 984.722076] LNet: 56442:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 984.725921] LNetError: 56442:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 984.733384] LNet: Removed LNI 192.168.203.107@tcp [ 985.163164] Key type .llcrypt unregistered [ 985.164837] Key type ._llcrypt unregistered [ 997.794401] Key type ._llcrypt registered [ 997.796613] Key type .llcrypt registered [ 997.844343] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 998.397715] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 998.413861] alg: No test for adler32 (adler32-zlib) [ 999.303058] Lustre: Lustre: Build Version: 2.17.57_82_g90dbd01 [ 999.417960] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 1001.015202] Key type lgssc registered [ 1001.472940] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1004.671146] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1004.681864] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1005.928285] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1007.746476] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1011.127288] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1014.122670] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1014.330800] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1015.336133] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:96 to 0x240000400:129) [ 1015.372288] Lustre: 59150:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1015.379583] Lustre: 59150:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1015.384813] Lustre: 59150:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1015.388590] Lustre: 59150:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1015.391724] Lustre: 59150:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1015.394405] Lustre: 59150:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1016.683381] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1019.622986] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1022.876099] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1025.014445] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1056.224569] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1056.225141] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1056.236063] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1061.344870] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1062.239343] Lustre: server umount lustre-OST0000 complete [ 1063.926301] Lustre: server umount lustre-MDT0000 complete [ 1068.391566] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 1069.633594] Key type lgssc unregistered [ 1069.792623] LNet: 60786:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1069.796454] LNetError: 60786:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1069.803400] LNet: Removed LNI 192.168.203.107@tcp [ 1070.171991] Key type .llcrypt unregistered [ 1070.173593] Key type ._llcrypt unregistered [ 1082.917298] Key type ._llcrypt registered [ 1082.918954] Key type .llcrypt registered [ 1082.967941] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 1083.407485] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1083.419808] alg: No test for adler32 (adler32-zlib) [ 1084.321992] Lustre: Lustre: Build Version: 2.17.57_82_g90dbd01 [ 1084.440312] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 1086.032713] Key type lgssc registered [ 1086.550686] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1089.883837] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1089.897462] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1091.105623] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1092.813994] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1095.722319] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1098.357278] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1098.503293] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1100.582573] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1100.608305] Lustre: 63593:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1100.612425] Lustre: 63593:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1100.615433] Lustre: 63593:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1100.618440] Lustre: 63593:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1100.621453] Lustre: 63593:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1100.624600] Lustre: 63593:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1100.673160] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1103.405141] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1106.484674] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1111.008523] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1111.010988] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1111.021678] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1115.421466] Lustre: server umount lustre-OST0000 complete [ 1117.398972] Lustre: server umount lustre-MDT0000 complete [ 1126.754483] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 1129.415966] Key type lgssc unregistered [ 1129.647281] LNet: 65152:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1129.652404] LNetError: 65152:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1129.662820] LNet: Removed LNI 192.168.203.107@tcp [ 1130.318423] Key type .llcrypt unregistered [ 1130.320799] Key type ._llcrypt unregistered [ 1148.319200] Key type ._llcrypt registered [ 1148.321618] Key type .llcrypt registered [ 1148.405194] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [ 1165.392159] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 1167.013466] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1167.056649] alg: No test for adler32 (adler32-zlib) [ 1168.152795] Lustre: Lustre: Build Version: 2.17.57_82_g90dbd01 [ 1168.417732] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 1170.279522] Key type lgssc registered [ 1173.782521] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1206.152800] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1206.164881] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1207.351630] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1207.372662] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1207.432534] Lustre: lustre-MDT0000: new disk, initializing [ 1207.476084] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1207.489041] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1210.487307] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1217.279796] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1217.350305] Lustre: 68307:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1217.398200] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1217.402262] Lustre: Skipped 1 previous similar message [ 1217.457124] Lustre: lustre-MDT0001: new disk, initializing [ 1217.521313] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1217.559533] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1217.566684] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1220.638219] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1227.042932] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1230.464385] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1234.634505] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1234.839459] Lustre: lustre-OST0000: new disk, initializing [ 1234.846112] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1234.850983] Lustre: 70274:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1234.903728] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1238.737327] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1243.710494] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1243.717646] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1243.799057] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1244.977203] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1248.107620] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1248.256707] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1251.281874] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1251.432038] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1254.372329] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1254.373123] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1254.377201] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1254.379556] LustreError: Skipped 1 previous similar message [ 1254.390376] Lustre: Skipped 1 previous similar message [ 1259.493225] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1259.497593] Lustre: Skipped 2 previous similar messages [ 1264.608834] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1264.615154] Lustre: Skipped 1 previous similar message [ 1267.167232] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1267.258367] Lustre: server umount lustre-OST0000 complete [ 1269.728684] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1269.742375] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1269.746943] Lustre: Skipped 1 previous similar message [ 1274.849446] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1274.861662] Lustre: Skipped 1 previous similar message [ 1275.960978] Lustre: server umount lustre-MDT0000 complete [ 1278.646902] LustreError: 68298:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788270472 with bad export cookie 9156547279639979389 [ 1278.655487] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1278.661510] LustreError: 68298:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1278.871425] Lustre: server umount lustre-MDT0001 complete [ 1284.454653] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 09:47:57 (1788270477) [ 1291.733530] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 1316.793365] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [ 1324.587884] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 1349.885974] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1350.058182] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1350.092769] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1350.151664] Lustre: lustre-MDT0000: new disk, initializing [ 1350.233920] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1350.256144] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1353.177548] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1361.078534] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1361.217299] Lustre: 76290:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1361.267925] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1361.271135] Lustre: Skipped 1 previous similar message [ 1361.345395] Lustre: lustre-MDT0001: new disk, initializing [ 1361.424083] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1361.446808] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1361.460074] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1364.555941] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1371.897921] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1376.283607] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1381.516693] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1381.802225] Lustre: lustre-OST0000: new disk, initializing [ 1381.811345] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1381.823619] Lustre: 78257:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1381.915975] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1387.030436] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1388.079064] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1388.094659] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1388.188802] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1394.451755] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1397.554106] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1397.751619] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1400.947427] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1401.132207] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1403.875985] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1403.876355] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1403.887671] LustreError: Skipped 1 previous similar message [ 1403.913280] Lustre: Skipped 1 previous similar message [ 1403.923397] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1403.931478] Lustre: Skipped 1 previous similar message [ 1408.785429] Lustre: server umount lustre-OST0000 complete [ 1412.576846] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1412.588453] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1412.605481] Lustre: Skipped 1 previous similar message [ 1418.240071] Lustre: server umount lustre-MDT0000 complete [ 1421.330343] LustreError: 76282:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788270615 with bad export cookie 9156547279639980271 [ 1421.334846] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1421.347481] LustreError: 76282:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1421.533425] Lustre: server umount lustre-MDT0001 complete [ 1430.415305] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 09:50:22 (1788270622) [ 1437.139788] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1437.415337] LustreError: 80669:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1437.432437] LustreError: 80669:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 1437.475321] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1442.047676] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1442.788093] LustreError: 80670:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1447.909163] LustreError: 80669:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1449.461765] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1453.623830] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1459.555800] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1464.058803] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1469.037611] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1473.749643] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1478.790955] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1481.586203] Lustre: 83340:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting f2395a32-65fd-451d-83a9-3d4cc3a8d2db at adminstrative request [ 1482.662318] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1482.676383] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 1492.965234] Lustre: MGS: Client f2395a32-65fd-451d-83a9-3d4cc3a8d2db (at 0@lo) reconnecting [ 1492.980237] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x7f12973cc402ac5a to 0x7f12973cc402af01 [ 1493.006035] Lustre: MGC192.168.203.107@tcp: Connection restored to 0@lo (at 0@lo) [ 1516.877371] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1517.026250] Lustre: lustre-OST0001: new disk, initializing [ 1517.030793] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1517.043768] Lustre: 83732:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1517.111086] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1517.115202] Lustre: Skipped 2 previous similar messages [ 1522.191487] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1527.349902] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1527.360643] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1527.436665] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1529.689663] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1532.384053] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1532.391832] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1532.402732] Lustre: Skipped 1 previous similar message [ 1532.409621] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 1532.415352] Lustre: Skipped 3 previous similar messages [ 1534.081381] Lustre: server umount lustre-OST0001 complete [ 1538.026825] LustreError: 83353:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1538.040246] LustreError: 83353:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 1543.143109] LustreError: 83353:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1543.155150] LustreError: 83353:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 1551.327141] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1551.466964] Lustre: server umount lustre-OST0000 complete [ 1557.476221] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1557.484206] LustreError: Skipped 1 previous similar message [ 1557.489253] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1557.498204] Lustre: Skipped 3 previous similar messages [ 1560.796538] Lustre: server umount lustre-MDT0000 complete [ 1563.301521] LustreError: 80649:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788270757 with bad export cookie 9156547279639981825 [ 1563.307030] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1563.309188] LustreError: 80649:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1563.510096] Lustre: server umount lustre-MDT0001 complete [ 1569.709149] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 09:52:42 (1788270762) [ 1571.197729] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 1572.762445] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 09:52:45 (1788270765) [ 1573.970828] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 1575.349820] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 09:52:48 (1788270768) [ 1576.584308] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 1577.980890] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 09:52:51 (1788270771) [ 1579.330741] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 1580.915217] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 09:52:53 (1788270773) [ 1582.068677] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 1585.961974] loop: module loaded [ 1585.976219] loop0: detected capacity change from 0 to 4294967296 [ 1586.020199] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 1587.241859] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 1589.449237] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 1592.424564] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 1610.945752] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1610.975635] LustreError: 87857:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 1610.988344] LustreError: 87857:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 1610.997833] LustreError: 87857:0:(tgt_mount.c:2582:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 1611.012811] LustreError: 87857:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 1624.586597] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [ 1631.993778] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 1655.323760] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1655.487839] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1655.511979] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1655.569568] Lustre: lustre-MDT0000: new disk, initializing [ 1655.603559] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1655.622107] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1658.252312] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1664.513997] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1664.575622] Lustre: 90719:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1664.726244] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1666.876729] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1672.122806] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1675.426213] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1679.188312] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1679.328075] Lustre: 92686:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1682.943859] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1686.549726] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1686.643664] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1689.345501] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1692.286765] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1692.435132] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1695.285114] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1695.443396] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1696.738044] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1696.739940] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1696.740280] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1696.740292] Lustre: Skipped 10 previous similar messages [ 1696.748760] Lustre: Skipped 2 previous similar messages [ 1696.753015] LustreError: Skipped 1 previous similar message [ 1702.566613] Lustre: server umount lustre-OST0000 complete [ 1710.636533] Lustre: server umount lustre-MDT0000 complete [ 1712.097838] LustreError: 92187:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1712.107409] LustreError: 92187:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 1712.475442] LustreError: 90711:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788270906 with bad export cookie 9156547279639982441 [ 1712.476407] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1712.482721] LustreError: 90711:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1712.630908] Lustre: server umount lustre-MDT0001 complete [ 1716.461178] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 09:55:09 (1788270909) [ 1719.528582] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1721.922863] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1726.167103] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1728.191805] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1731.572814] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1734.184893] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1737.505069] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1740.835857] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1744.583820] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1746.781424] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1746.929967] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1748.934444] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1752.138740] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 3 sec [ 1755.550447] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1755.642414] Lustre: lustre-OST0001: new disk, initializing [ 1755.645550] Lustre: Skipped 2 previous similar messages [ 1755.649665] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1755.652313] Lustre: Skipped 3 previous similar messages [ 1755.655210] Lustre: 98481:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1758.429989] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1763.024807] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1764.385621] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1764.389916] Lustre: Skipped 2 previous similar messages [ 1764.394816] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1764.439110] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1765.052956] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 1765.181706] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1767.001668] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 1767.101449] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1779.849314] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 1791.423409] Lustre: Failing over lustre-MDT0000 [ 1791.803692] Lustre: server umount lustre-MDT0000 complete [ 1792.992635] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1793.001882] LustreError: Skipped 1 previous similar message [ 1793.004334] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1793.011538] Lustre: Skipped 2 previous similar messages [ 1793.015274] LustreError: 95093:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1793.022552] LustreError: 95093:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 1807.543952] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 1807.546127] LDISKFS-fs (dm-0): recovery complete [ 1807.550716] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1807.609918] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1807.741811] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 1807.744709] Lustre: Skipped 6 previous similar messages [ 1807.772141] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 1809.272646] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1810.458369] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 1810.459034] Lustre: 95088:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1812.962731] Lustre: 95093:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1812.967016] Lustre: 95093:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 1812.972814] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 1812.995256] LustreError: 101303:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1813.311125] LustreError: 101303:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1813.626113] LustreError: 101303:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1813.630866] LustreError: 101303:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 1813.943108] LustreError: 101303:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1813.947165] LustreError: 101303:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 1814.265329] Lustre: 101303:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1814.271433] Lustre: 101303:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 8 previous similar messages [ 1814.897524] LustreError: 101303:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1814.902306] LustreError: 101303:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 3 previous similar messages [ 1815.215200] LustreError: 101303:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1815.219524] LustreError: 101303:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 3 previous similar messages [ 1816.471946] Lustre: 101303:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1816.478304] Lustre: 101303:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 6 previous similar messages [ 1817.112160] LustreError: 101303:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1817.116759] LustreError: 101303:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 1817.431093] LustreError: 101303:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1817.434472] LustreError: 101303:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 1820.563465] Lustre: 101303:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1820.568676] Lustre: 101303:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 1821.209265] LustreError: 101303:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1821.213608] LustreError: 101303:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 1821.527306] LustreError: 101303:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1821.538729] LustreError: 101303:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 1828.737315] Lustre: 101303:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1828.742580] Lustre: 101303:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 25 previous similar messages [ 1829.369386] LustreError: 101303:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1829.373774] LustreError: 101303:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 25 previous similar messages [ 1829.687114] LustreError: 101303:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1829.690138] LustreError: 101303:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 25 previous similar messages [ 1844.993446] Lustre: 101303:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1844.998738] Lustre: 101303:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 51 previous similar messages [ 1845.625157] LustreError: 101303:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1845.629732] LustreError: 101303:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 51 previous similar messages [ 1845.944055] LustreError: 101303:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1845.946917] LustreError: 101303:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 51 previous similar messages [ 1870.287180] Lustre: 101303:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 1870.291970] Lustre: 101303:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 1870.296652] Lustre: 101303:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 629566a8-cfd1-40a8-a48d-b0387d48109b@192.168.203.7@tcp [ 1870.302284] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 1870.305565] Lustre: 101303:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@00000000164ed17e x1875137365348480/t0(8589934780) o35->629566a8-cfd1-40a8-a48d-b0387d48109b@192.168.203.7@tcp:724/0 lens 392/0 e 0 to 0 dl 1788271074 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 1870.316943] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 1870.320482] Lustre: lustre-MDT0000: Denying connection for new client 629566a8-cfd1-40a8-a48d-b0387d48109b (at 192.168.203.7@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 1870.326074] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 1870.341882] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1870.345578] Lustre: Skipped 2 previous similar messages [ 1870.348831] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 1870.364765] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:80 to 0x280000401:577) [ 1870.364765] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:48 to 0x2c0000401:545) [ 1875.935902] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1875.939616] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1875.944446] Lustre: Skipped 3 previous similar messages [ 1875.946738] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1875.949112] Lustre: Skipped 6 previous similar messages [ 1877.247673] Lustre: server umount lustre-OST0000 complete [ 1879.523245] LustreError: 96914:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1879.529916] LustreError: 96914:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 19 previous similar messages [ 1884.631461] Lustre: server umount lustre-OST0001 complete [ 1892.073937] Lustre: server umount lustre-MDT0000 complete [ 1893.328387] LustreError: 95073:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788271087 with bad export cookie 9156547279640033653 [ 1893.331610] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1893.333111] LustreError: 95073:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1893.454780] Lustre: server umount lustre-MDT0001 complete [ 1896.763207] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 09:58:10 (1788271090) [ 1899.490964] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1899.494566] Lustre: *** cfs_fail_loc=197, val=0*** [ 1914.911575] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 1920.031552] LustreError: 103273:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.107@tcp: failed processing log, type 4: rc = -110 [ 1925.157523] Lustre: *** cfs_fail_loc=197, val=0*** [ 1925.159230] Lustre: Skipped 65 previous similar messages [ 1947.536740] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1949.990267] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1950.677374] Lustre: server umount lustre-OST0000 complete [ 1953.489223] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 09:59:07 (1788271147) [ 1967.133690] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1967.291742] LustreError: 104890:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1968.756878] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1972.570361] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1974.602710] LustreError: 104924:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 1974.606824] LustreError: 104924:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 1974.610469] LustreError: 104924:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -5 [ 1974.727566] Lustre: server umount lustre-MDT0000 complete [ 1987.461860] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [ 1991.619349] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 2005.561976] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2005.669847] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2005.683458] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2005.724841] Lustre: lustre-MDT0000: new disk, initializing [ 2005.757220] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2007.359023] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2011.823980] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2011.865433] Lustre: 109087:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2011.976259] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2013.403672] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2017.264271] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2019.009864] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2021.156946] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2021.265032] Lustre: 111055:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2023.062190] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2026.605038] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2028.097321] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2030.104877] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2030.219046] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 2 sec [ 2031.724270] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2031.827447] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2035.169647] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2035.170107] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2035.172626] LustreError: Skipped 1 previous similar message [ 2035.176808] Lustre: Skipped 5 previous similar messages [ 2035.177948] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2035.182097] Lustre: Skipped 5 previous similar messages [ 2038.542782] Lustre: server umount lustre-OST0000 complete [ 2054.111144] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2054.172914] Lustre: server umount lustre-MDT0000 complete [ 2055.429115] LustreError: 109079:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788271249 with bad export cookie 9156547279640044356 [ 2055.431782] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2055.434085] LustreError: 109079:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2055.549964] Lustre: server umount lustre-MDT0001 complete [ 2058.494114] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 10:00:52 (1788271252) [ 2061.778345] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 2062.878682] Key type lgssc unregistered [ 2063.004446] LNet: 113559:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2063.008438] LNetError: 113559:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2063.015516] LNet: Removed LNI 192.168.203.107@tcp [ 2063.403112] Key type .llcrypt unregistered [ 2063.404767] Key type ._llcrypt unregistered [ 2069.720550] Key type ._llcrypt registered [ 2069.721911] Key type .llcrypt registered [ 2069.761859] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [ 2076.348548] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 2076.720921] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2076.771069] alg: No test for adler32 (adler32-zlib) [ 2077.632568] Lustre: Lustre: Build Version: 2.17.57_82_g90dbd01 [ 2077.729627] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 2079.311208] Key type lgssc registered [ 2079.699449] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2093.137834] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2093.145350] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2094.227789] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2094.239961] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2094.278698] Lustre: lustre-MDT0000: new disk, initializing [ 2094.300236] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2094.305946] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2095.561553] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2099.696924] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2099.806849] Lustre: lustre-OST0000: new disk, initializing [ 2099.809283] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2099.811599] Lustre: Skipped 1 previous similar message [ 2099.813534] Lustre: 117717:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2099.834769] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2101.677667] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2102.462171] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 2102.464875] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 2102.473614] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 2105.714519] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2105.769104] Lustre: lustre-OST0001: new disk, initializing [ 2105.770589] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2105.772776] Lustre: 118578:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2105.795745] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2107.649291] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2111.495335] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 2111.499748] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 2111.507211] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 2111.707157] Lustre: server umount lustre-MDT0000 complete [ 2113.662397] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 2115.189199] LustreError: 116902:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788271309 with bad export cookie 9652537372350061148 [ 2115.194798] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2125.821230] Lustre: server umount lustre-OST0000 complete [ 2136.224455] Lustre: server umount lustre-OST0001 complete [ 2138.899787] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 10:02:12 (1788271332) [ 2146.094297] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 10:02:19 (1788271339) [ 2149.043016] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 2155.833667] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 10:02:29 (1788271349) [ 2167.272666] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [ 2170.106444] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 2181.283017] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2181.355730] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2181.367106] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2181.397519] Lustre: lustre-MDT0000: new disk, initializing [ 2181.417036] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2181.422938] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2182.620485] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2186.491523] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2186.531703] Lustre: 126326:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2186.552882] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2186.554551] Lustre: Skipped 1 previous similar message [ 2186.588512] Lustre: lustre-MDT0001: new disk, initializing [ 2186.615211] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2186.624947] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2186.629746] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2187.776545] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2191.438898] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2193.349826] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2195.594918] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2195.702669] Lustre: lustre-OST0000: new disk, initializing [ 2195.704340] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2195.707088] Lustre: 128290:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2195.738157] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2197.802530] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2201.558072] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2202.989255] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2203.127863] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2203.132761] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2203.159701] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2204.083102] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 1 sec [ 2205.540219] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2205.614209] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2208.227071] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2208.228045] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2208.230132] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2208.235988] Lustre: Skipped 1 previous similar message [ 2212.366627] Lustre: server umount lustre-OST0000 complete [ 2217.439921] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2217.442028] LustreError: Skipped 1 previous similar message [ 2217.443216] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2217.446850] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2217.448676] Lustre: Skipped 1 previous similar message [ 2218.464212] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2218.465020] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2219.684042] Lustre: server umount lustre-MDT0000 complete [ 2220.927332] LustreError: 126318:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788271414 with bad export cookie 9652537372350062352 [ 2220.930052] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2220.931413] LustreError: 126318:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2221.057618] Lustre: server umount lustre-MDT0001 complete [ 2223.885445] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 10:03:37 (1788271417) [ 2226.053801] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2226.252496] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2227.620322] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2230.330100] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2231.650055] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2234.133417] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2236.206389] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2239.467335] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2244.786813] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2252.944418] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2257.458518] Lustre: *** cfs_fail_loc=159, val=0*** [ 2272.886988] Lustre: lustre-MDT0000: Client 5e777dca-e22b-48cb-b0af-f1bb50c70581 (at 192.168.203.7@tcp) reconnecting [ 2274.059803] Lustre: *** cfs_fail_loc=159, val=0*** [ 2274.067862] Lustre: Skipped 5 previous similar messages [ 2290.229888] Lustre: lustre-MDT0000: Client 5e777dca-e22b-48cb-b0af-f1bb50c70581 (at 192.168.203.7@tcp) reconnecting [ 2307.552736] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2307.557533] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2307.565423] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2307.569185] Lustre: Skipped 1 previous similar message [ 2312.184025] Lustre: server umount lustre-OST0000 complete [ 2317.791933] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2317.795571] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2317.801155] Lustre: Skipped 1 previous similar message [ 2317.803744] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2317.809054] Lustre: Skipped 2 previous similar messages [ 2320.161955] Lustre: server umount lustre-MDT0000 complete [ 2322.142206] LustreError: 133427:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788271515 with bad export cookie 9652537372350063206 [ 2322.145056] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2322.146747] LustreError: 133427:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2322.298944] Lustre: server umount lustre-MDT0001 complete [ 2331.948210] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 2335.271515] Key type lgssc unregistered [ 2335.542787] LNet: 134485:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2335.549776] LNetError: 134485:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2335.561886] LNet: Removed LNI 192.168.203.107@tcp [ 2336.142917] Key type .llcrypt unregistered [ 2336.144642] Key type ._llcrypt unregistered [ 2364.230848] Key type ._llcrypt registered [ 2364.237348] Key type .llcrypt registered [ 2364.360374] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 2365.715538] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2365.739710] alg: No test for adler32 (adler32-zlib) [ 2366.863632] Lustre: Lustre: Build Version: 2.17.57_82_g90dbd01 [ 2367.241485] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 2368.975497] Key type lgssc registered [ 2370.569879] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2380.112422] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2380.128524] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2381.543029] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2385.422510] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2392.036189] LustreError: 135901:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2392.055862] LustreError: 135901:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 2392.203575] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2392.538881] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2397.092922] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2405.115642] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2410.413787] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2416.404992] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2416.609136] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2421.083996] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2426.289792] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2426.874741] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 2435.445194] Lustre: *** cfs_fail_loc=159, val=0*** [ 2451.015517] Lustre: lustre-MDT0000: Client 40a0af09-634f-4848-ae82-6d39dc893f44 (at 192.168.203.7@tcp) reconnecting [ 2452.918737] Lustre: *** cfs_fail_loc=159, val=0*** [ 2452.923570] Lustre: Skipped 1 previous similar message [ 2469.467140] Lustre: lustre-MDT0001: Client 40a0af09-634f-4848-ae82-6d39dc893f44 (at 192.168.203.7@tcp) reconnecting [ 2470.711897] Lustre: *** cfs_fail_loc=159, val=0*** [ 2470.717191] Lustre: Skipped 2 previous similar messages [ 2486.367987] Lustre: lustre-MDT0001: Client 40a0af09-634f-4848-ae82-6d39dc893f44 (at 192.168.203.7@tcp) reconnecting [ 2490.476660] Lustre: 138940:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2492.797495] Lustre: *** cfs_fail_loc=159, val=0*** [ 2508.363440] Lustre: lustre-MDT0000: Client 84847604-94da-4b09-9ae7-23b6e383ce4a (at 192.168.203.7@tcp) reconnecting [ 2509.929534] Lustre: *** cfs_fail_loc=159, val=0*** [ 2509.931346] Lustre: Skipped 14 previous similar messages [ 2526.295180] Lustre: lustre-MDT0000: Client 84847604-94da-4b09-9ae7-23b6e383ce4a (at 192.168.203.7@tcp) reconnecting [ 2527.630371] Lustre: 139230:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2549.741850] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2549.748379] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2549.761393] Lustre: Skipped 1 previous similar message [ 2549.762109] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2549.782116] Lustre: Skipped 1 previous similar message [ 2552.002534] Lustre: server umount lustre-OST0000 complete [ 2556.384217] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2556.389654] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2556.408620] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2559.970401] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2559.971971] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2565.088895] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2565.096899] Lustre: Skipped 2 previous similar messages [ 2569.192313] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2569.203113] Lustre: Skipped 1 previous similar message [ 2570.207125] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2570.387391] Lustre: server umount lustre-MDT0000 complete [ 2573.696656] LustreError: 139062:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788271767 with bad export cookie 8853067094780509079 [ 2573.698378] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2573.720895] LustreError: 139062:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2574.155331] Lustre: server umount lustre-MDT0001 complete [ 2585.287426] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 2588.950505] Key type lgssc unregistered [ 2589.231843] LNet: 140313:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2589.250245] LNetError: 140313:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2589.284177] LNet: Removed LNI 192.168.203.107@tcp [ 2590.035623] Key type .llcrypt unregistered [ 2590.040081] Key type ._llcrypt unregistered [ 2615.728428] Key type ._llcrypt registered [ 2615.736315] Key type .llcrypt registered [ 2615.862882] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 2616.800593] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2616.810422] alg: No test for adler32 (adler32-zlib) [ 2617.959298] Lustre: Lustre: Build Version: 2.17.57_82_g90dbd01 [ 2618.232030] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 2619.951168] Key type lgssc registered [ 2621.132188] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2629.214501] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2629.235293] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2630.549697] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2635.351810] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2640.865326] LustreError: 141737:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2640.891067] LustreError: 141737:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 2642.707819] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2642.930874] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2646.625770] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2653.515744] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2659.100163] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2664.783738] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2665.091053] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2669.185885] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 2671.163856] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2675.339663] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 2678.339381] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2684.070432] Lustre: 144317:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2686.991608] Lustre: 144365:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2688.366318] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 2689.189727] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 2692.893112] Lustre: 144511:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 2693.721643] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 2694.451457] Lustre: 144609:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2700.260034] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2700.266938] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2700.278945] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2701.283367] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2701.293736] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2701.308214] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2704.187344] Lustre: server umount lustre-OST0000 complete [ 2709.472969] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2709.481741] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2709.493575] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2709.499656] Lustre: Skipped 1 previous similar message [ 2711.523234] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2711.524978] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2713.825746] Lustre: server umount lustre-MDT0000 complete [ 2716.640446] LustreError: 143073:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2716.659320] LustreError: 143073:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2717.036147] LustreError: 142472:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788271910 with bad export cookie 10833950805356964151 [ 2717.037259] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2717.048652] LustreError: 142472:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2717.332924] Lustre: server umount lustre-MDT0001 complete [ 2727.838135] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 2730.258778] Key type lgssc unregistered [ 2730.500690] LNet: 145674:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2730.505619] LNetError: 145674:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2730.519581] LNet: Removed LNI 192.168.203.107@tcp [ 2731.259787] Key type .llcrypt unregistered [ 2731.261907] Key type ._llcrypt unregistered [ 2753.980780] Key type ._llcrypt registered [ 2753.982614] Key type .llcrypt registered [ 2754.085788] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 2754.962769] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2755.119787] alg: No test for adler32 (adler32-zlib) [ 2756.246204] Lustre: Lustre: Build Version: 2.17.57_82_g90dbd01 [ 2756.528067] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 2758.200106] Key type lgssc registered [ 2759.404925] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2767.303607] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2767.333454] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2768.875491] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2773.521556] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2779.114025] LustreError: 147089:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2779.139682] LustreError: 147089:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 2781.841701] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2782.182838] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2786.234363] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2793.528530] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2798.537261] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2804.092197] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2804.483238] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2806.592745] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 2810.331878] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2813.816336] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 2816.603285] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2823.002484] Lustre: *** cfs_fail_loc=159, val=0*** [ 2839.651034] Lustre: lustre-MDT0000: Client 14d5c961-4db4-4fd5-b3f3-cb4a9b1dc163 (at 192.168.203.7@tcp) reconnecting [ 2845.158558] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 2845.159363] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2845.171586] LustreError: Skipped 1 previous similar message [ 2845.190802] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2846.855164] Lustre: server umount lustre-OST0000 complete [ 2853.856049] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2853.863031] LustreError: Skipped 2 previous similar messages [ 2853.869136] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2853.896136] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2854.883684] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2854.902378] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2854.910540] Lustre: Skipped 1 previous similar message [ 2856.277243] Lustre: server umount lustre-MDT0000 complete [ 2859.482607] LustreError: 147071:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788272053 with bad export cookie 6172966881707393972 [ 2859.487054] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2859.497477] LustreError: 147071:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2869.881400] Lustre: server umount lustre-MDT0001 complete [ 2878.432760] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 2880.564863] Key type lgssc unregistered [ 2880.828630] LNet: 150754:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2880.837957] LNetError: 150754:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2881.899694] LNet: Removed LNI 192.168.203.107@tcp [ 2882.502629] Key type .llcrypt unregistered [ 2882.504430] Key type ._llcrypt unregistered [ 2903.781505] Key type ._llcrypt registered [ 2903.784589] Key type .llcrypt registered [ 2903.885422] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 2904.835779] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2904.898886] alg: No test for adler32 (adler32-zlib) [ 2905.923974] Lustre: Lustre: Build Version: 2.17.57_82_g90dbd01 [ 2906.167220] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 2907.847199] Key type lgssc registered [ 2908.920331] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2914.881229] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2914.906179] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2916.231235] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2919.671512] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2926.564762] LustreError: 152171:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2926.576738] LustreError: 152171:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 2926.938945] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2927.270280] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2930.802461] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2937.426264] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2942.741974] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2948.403037] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2948.740687] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2952.818930] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 2953.914589] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2959.989116] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2962.033370] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 2981.197988] Lustre: lustre-OST0000: evicting 77c12688-da27-46a9-8384-a4f2dee81804 (at 192.168.203.7@tcp) by administrative request [ 2981.210281] Lustre: Skipped 1 previous similar message [ 3008.489381] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3008.490670] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3008.491244] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3008.504084] Lustre: Skipped 1 previous similar message [ 3008.518237] Lustre: Skipped 1 previous similar message [ 3013.503742] Lustre: server umount lustre-OST0000 complete [ 3018.724624] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3018.731957] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3018.740506] Lustre: Skipped 1 previous similar message [ 3018.750757] Lustre: Skipped 1 previous similar message [ 3022.370866] Lustre: server umount lustre-MDT0000 complete [ 3023.843565] LustreError: 152171:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3023.854660] LustreError: 152171:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3024.692201] LustreError: 152150:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788272218 with bad export cookie 6520894784481880064 [ 3024.693674] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3024.700522] LustreError: 152150:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3024.951415] Lustre: server umount lustre-MDT0001 complete [ 3032.506928] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 3035.045097] Key type lgssc unregistered [ 3035.320506] LNet: 156240:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3035.326244] LNetError: 156240:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3035.336577] LNet: Removed LNI 192.168.203.107@tcp [ 3035.927942] Key type .llcrypt unregistered [ 3035.930180] Key type ._llcrypt unregistered [ 3088.670579] Key type ._llcrypt registered [ 3088.673250] Key type .llcrypt registered [ 3088.755579] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 3089.210782] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3089.253640] alg: No test for adler32 (adler32-zlib) [ 3090.252541] Lustre: Lustre: Build Version: 2.17.57_82_g90dbd01 [ 3090.443419] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 3092.071127] Key type lgssc registered [ 3092.708713] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3098.634967] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3098.654422] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3100.016606] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3106.265715] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 3109.349952] LustreError: 158707:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3109.368613] LustreError: 158707:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3114.466549] LustreError: 158706:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3119.143625] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3119.606203] LustreError: 158707:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3120.133427] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3125.183803] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 3138.866274] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3146.517568] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3153.527509] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3153.940591] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3159.193163] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 3164.213183] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 3166.952528] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3169.322988] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 3184.611376] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3184.612301] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3184.628036] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3184.680397] Lustre: Skipped 1 previous similar message [ 3186.660646] Lustre: server umount lustre-OST0000 complete [ 3194.867910] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3194.876543] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3194.886020] Lustre: Skipped 1 previous similar message [ 3194.889939] Lustre: Skipped 1 previous similar message [ 3199.991589] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3200.000536] Lustre: Skipped 2 previous similar messages [ 3200.292806] Lustre: server umount lustre-MDT0000 complete [ 3204.541075] LustreError: 158734:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788272398 with bad export cookie 2051366126816737388 [ 3204.544105] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3204.562579] LustreError: 158734:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3204.897815] Lustre: server umount lustre-MDT0001 complete [ 3217.594357] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 3221.137180] Key type lgssc unregistered [ 3221.349627] LNet: 162275:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3221.354805] LNetError: 162275:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3222.373216] LNet: Removed LNI 192.168.203.107@tcp [ 3223.192828] Key type .llcrypt unregistered [ 3223.195065] Key type ._llcrypt unregistered [ 3252.552724] Key type ._llcrypt registered [ 3252.556610] Key type .llcrypt registered [ 3252.680633] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 3254.118367] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3254.156771] alg: No test for adler32 (adler32-zlib) [ 3255.189200] Lustre: Lustre: Build Version: 2.17.57_82_g90dbd01 [ 3255.498707] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 3257.271347] Key type lgssc registered [ 3258.389229] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3274.322380] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 10:21:06 (1788272466) [ 3291.085692] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [ 3301.917927] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 3340.074602] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3340.118945] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3341.351348] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3341.389924] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3341.490870] Lustre: lustre-MDT0000: new disk, initializing [ 3341.587665] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3341.619896] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3346.383821] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 3356.647823] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3356.787635] Lustre: 166538:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3356.854903] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3356.859304] Lustre: Skipped 1 previous similar message [ 3356.984427] Lustre: lustre-MDT0001: new disk, initializing [ 3357.110946] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3357.152970] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3357.168437] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3361.282824] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 3369.596568] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3374.804226] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3381.223493] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3381.461724] Lustre: lustre-OST0000: new disk, initializing [ 3381.467508] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3381.482986] Lustre: 168507:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3381.582994] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3387.643591] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 3389.555302] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3389.587493] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3389.721846] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3398.359790] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3410.409525] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3410.411363] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3410.435978] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3410.440914] Lustre: Skipped 1 previous similar message [ 3414.558691] Lustre: server umount lustre-OST0000 complete [ 3420.645278] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3420.670650] Lustre: Skipped 1 previous similar message [ 3420.679715] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3420.686693] Lustre: Skipped 1 previous similar message [ 3425.076403] Lustre: server umount lustre-MDT0000 complete [ 3428.832663] LustreError: 166529:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788272622 with bad export cookie 15691470174565028471 [ 3428.848894] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3429.362277] Lustre: server umount lustre-MDT0001 complete [ 3440.909739] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 3443.892501] Key type lgssc unregistered [ 3444.140086] LNet: 170416:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3444.154776] LNetError: 170416:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3444.176852] LNet: Removed LNI 192.168.203.107@tcp [ 3444.985153] Key type .llcrypt unregistered [ 3444.994077] Key type ._llcrypt unregistered [ 3471.242224] Key type ._llcrypt registered [ 3471.248441] Key type .llcrypt registered [ 3471.368932] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 3472.485239] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3472.504638] alg: No test for adler32 (adler32-zlib) [ 3473.586914] Lustre: Lustre: Build Version: 2.17.57_82_g90dbd01 [ 3473.867834] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 3475.559195] Key type lgssc registered [ 3476.871414] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3484.368873] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3484.406239] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3485.748658] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3489.660725] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 3495.908841] LustreError: 171833:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3495.928742] LustreError: 171833:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3497.873808] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3498.266623] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3502.805568] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 3511.349852] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3517.441463] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3524.346891] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3524.651412] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3530.216843] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 3530.810222] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 3539.265289] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3548.462616] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3548.498400] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3548.508464] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3548.515560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3548.534451] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3549.420605] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3549.455569] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3549.481687] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3549.502354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3549.531101] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3550.585671] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3550.602694] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3550.614832] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3550.621928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3550.633270] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3551.422116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3552.363902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3553.215764] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3553.229133] Lustre: Skipped 2 previous similar messages [ 3553.249091] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3553.260502] Lustre: Skipped 2 previous similar messages [ 3553.264939] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3553.273308] Lustre: Skipped 2 previous similar messages [ 3553.281552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3553.299706] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3553.310479] Lustre: Skipped 2 previous similar messages [ 3554.398194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3555.403570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3556.250116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3557.095515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3558.097731] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3558.102571] Lustre: Skipped 4 previous similar messages [ 3558.116724] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3558.126924] Lustre: Skipped 4 previous similar messages [ 3558.130768] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3558.134374] Lustre: Skipped 4 previous similar messages [ 3558.137885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3558.144935] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3558.151149] Lustre: Skipped 4 previous similar messages [ 3558.974880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3559.909269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3560.656616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3561.662995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3562.579471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3563.685392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3564.625544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3565.651418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3566.723653] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3566.733169] Lustre: Skipped 8 previous similar messages [ 3566.758337] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3566.776927] Lustre: Skipped 8 previous similar messages [ 3566.781653] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3566.787645] Lustre: Skipped 8 previous similar messages [ 3566.791110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3566.804442] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3566.809356] Lustre: Skipped 8 previous similar messages [ 3567.811334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3569.093255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3570.082332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3570.986616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3572.011784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3572.950795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3574.060621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3575.237274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3576.174464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3577.252240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3578.090414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3579.108290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3579.926755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3580.825837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3581.713905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3582.676136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3583.602847] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3583.607071] Lustre: Skipped 16 previous similar messages [ 3583.622875] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3583.630145] Lustre: Skipped 16 previous similar messages [ 3583.633828] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3583.639496] Lustre: Skipped 16 previous similar messages [ 3583.642032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3583.651458] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3583.659474] Lustre: Skipped 16 previous similar messages [ 3584.559484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3585.388352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3586.251333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3587.142823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3587.966526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3588.823127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3589.616826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3590.537278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3591.326626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3592.281762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3593.212801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3594.118742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3594.958422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3595.955461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3596.976866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3597.953480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3598.840164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3599.649366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3600.448748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3601.208653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3602.138896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3602.996737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3603.830393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3604.645494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3605.392157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3606.252463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3607.166892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3608.060692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3608.903962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3609.876285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3610.662430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3611.553212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3612.430610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3613.269944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3614.190411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3615.133790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3615.991559] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3615.997625] Lustre: Skipped 36 previous similar messages [ 3616.026038] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3616.043385] Lustre: Skipped 36 previous similar messages [ 3616.050874] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3616.069770] Lustre: Skipped 36 previous similar messages [ 3616.083666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3616.096703] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3616.106955] Lustre: Skipped 36 previous similar messages [ 3617.025080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3617.993437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3618.996792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3619.924163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3620.790497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3621.734052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3622.579824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3623.335223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3624.080347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3624.779945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3625.561843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3626.163928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3626.939130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3627.653674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3628.480585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3629.327566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3630.214995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3630.946811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3631.735634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3632.576592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3633.317587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3634.094763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3634.899568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3635.624619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3636.437380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3637.226724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3637.988499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3638.786222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3639.826912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3640.842713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3641.684417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3642.550609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3643.491671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3644.259294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3645.040631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3645.737775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3646.519512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3647.235992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3648.104881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3648.983480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3649.965216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3650.879915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3651.633294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3652.713896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3653.544717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3654.510041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3655.238434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3656.190650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3657.131639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3657.946494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3658.760291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3659.611660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3660.442906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3661.298993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3662.179732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3662.996388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3663.983327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3664.966159] LustreError: 171853:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3664.985892] LustreError: 171853:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3664.993986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3665.782023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3666.647856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3667.551869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3668.489115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3669.469096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3670.465985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3671.432913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3672.366933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3673.240648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3674.143912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3675.005621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3675.825946] LustreError: 171853:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3675.843568] LustreError: 171853:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3675.858801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3676.599244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3677.619904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3678.512428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3679.469422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3680.258333] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3680.263253] Lustre: Skipped 74 previous similar messages [ 3680.283461] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3680.296805] Lustre: Skipped 74 previous similar messages [ 3680.301501] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3680.309766] Lustre: Skipped 74 previous similar messages [ 3680.321899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3680.335733] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3680.343982] Lustre: Skipped 74 previous similar messages [ 3681.375757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3682.228087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3683.091733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3683.929321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3684.849898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3685.763981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3686.638486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3687.561853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3688.513206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3689.337968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3690.242405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3691.222313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3692.114281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3693.110666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3693.933163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3694.767320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3695.779611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3696.763311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3697.710694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3698.471657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3699.520174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3700.549386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3701.522270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3702.491643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3703.501185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3704.379859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3705.315230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3706.219505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3707.271815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3708.180508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3709.085884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3709.978418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3710.851750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3711.566454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3712.634489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3713.529223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3714.600631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3715.607506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3716.426743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3717.277361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3718.130405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3718.986717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3719.984573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3720.789425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3721.703776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3722.710862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3723.707841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3724.630298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3725.411151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3726.232921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3727.220754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3728.082409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3728.965985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3729.880428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3730.687954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3731.545513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3732.387955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3733.194146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3734.054460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3734.929690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3735.802223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3736.606496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3737.390787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3738.346203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3739.257146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3740.080671] LustreError: 171853:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3740.096759] LustreError: 171853:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3740.109204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3740.933291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3741.740133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3742.548131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3743.343674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3744.209808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3745.093975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3745.791833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3746.734974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3747.457465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3748.263725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3748.998399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3749.973989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3750.936817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3751.854976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3752.758520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3753.767961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3754.621394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3755.543116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3756.400493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3757.152895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3757.961556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3758.763156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3759.570957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3760.341538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3761.189374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3762.014258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3762.809869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3763.619244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3764.385758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3765.392696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3766.136273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3767.087196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3768.096509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3768.978125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3769.708298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3770.688753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3771.365280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3772.340701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3773.155067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3774.072965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3774.787617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3775.761210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3776.627453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3777.587279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3778.422909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3779.432476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3780.503608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3781.369224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3782.532343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3783.416592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3784.439920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3785.557192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3786.519410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3787.409766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3788.630696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3789.612856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3790.696649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3791.644109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3792.803677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3793.818606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3795.069889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3796.285603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3797.162297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3798.220047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3798.975982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3799.809956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3800.744706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3801.473995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3802.271744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3803.182119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3803.994721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3805.049172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3805.909149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3806.917818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3807.789072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3808.671230] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3808.679215] Lustre: Skipped 141 previous similar messages [ 3808.698518] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3808.708453] Lustre: Skipped 141 previous similar messages [ 3808.712704] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3808.722910] Lustre: Skipped 141 previous similar messages [ 3808.726283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3808.742647] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3808.751296] Lustre: Skipped 141 previous similar messages [ 3809.469308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3810.443414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3811.336921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3812.282165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3813.042386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3814.079716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3815.114555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3816.504611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3817.382687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3818.512713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3819.495066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3820.584347] LustreError: 171853:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3820.591824] LustreError: 171853:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3820.615092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3821.532626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3822.431409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3823.311700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3824.585489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3825.846347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3826.916219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3827.828829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3828.822727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3829.766500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3830.491673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3831.283958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3831.982468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3832.825570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3833.484476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3834.386861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3835.309776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3836.331370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3837.345469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3838.290490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3839.445440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3840.659855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3841.655271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3842.679674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3843.750449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3844.764080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3846.105304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3847.183192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3848.401627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3849.460836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3850.459716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3851.569515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3852.773196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3853.912961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3854.976278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3856.214429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3857.133260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3858.195652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3859.171470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3860.260799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3861.216401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3862.303469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3863.384196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3864.387570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3865.365973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3866.457048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3867.378770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3868.451656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3869.462767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3870.472452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3871.374988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3872.608522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3873.621353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3874.880816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3875.973392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3877.067105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3878.230291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3879.212773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3880.173298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3881.130466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3882.036250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3882.893198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3883.834479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3884.772105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3885.664269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3886.547544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3887.578236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3888.581937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3889.524221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3890.497600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3891.785649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3892.912867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3893.801650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3894.771108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3895.850926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3896.732942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3897.739579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3898.708721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3899.684932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3900.843697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3902.119203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3903.506302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3904.594518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3905.638665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3906.684107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3907.824865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3908.791383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3910.017717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3911.200966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3912.308605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3913.520775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3914.475645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3915.497428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3916.461763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3917.464585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3918.407469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3919.418660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3920.347505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3921.511618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3922.503701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3923.468834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3924.343529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3925.379641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3926.224090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3927.123973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3927.893705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3928.843332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3929.756889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3930.674214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3931.714252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3932.665698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3933.710668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3934.713638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3935.660271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3936.526584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3937.536597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3938.630175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3939.925890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3941.241413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3942.273722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3943.348292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3944.627753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3945.723652] LustreError: 171853:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3945.748396] LustreError: 171853:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3945.757168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3946.947770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3947.830466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3948.891092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3950.163637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3951.173592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3952.438889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3953.649401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3954.744351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3955.772862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3956.842310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3957.731663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3958.682603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3959.637421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3960.873829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3961.846482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3963.301090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3964.592117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3965.541262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3966.562409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3967.481910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3968.246564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3969.176902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3970.136790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3971.028941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3972.011523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3972.924207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3974.005305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3975.028453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3975.994500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3976.783897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3977.703152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3978.901807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3979.894281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3980.921555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3981.787551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3982.910669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3984.055157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3984.992591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3986.128711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3987.086483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3988.074398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3988.876096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3989.799034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3990.887407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3991.807709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3993.028165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3994.223582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3995.460565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3996.883393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3997.933362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3999.128814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4000.114334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4001.080775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4002.250100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4003.229222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4004.531960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4005.582807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4006.697864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4007.757242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4008.657993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4009.578708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4010.584480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4011.775401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4012.629495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4013.623223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4014.608411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4015.492768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4016.310391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4017.160611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4018.068434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4018.940743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4019.800247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4020.722395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4021.607614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4022.498760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4023.447286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4024.398320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4025.381228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4026.183321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4027.112587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4028.098751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4028.935785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4029.712619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4030.691358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4031.540425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4032.377676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4033.292970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4034.131726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4034.909137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4035.811663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4036.758289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4037.598941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4038.373847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4039.206661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4039.981279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4040.977164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4041.849672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4042.691582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4043.494814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4044.401265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4045.189144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4045.972616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4046.816067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4047.723230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4048.558516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4049.330907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4050.028814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4050.844212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4051.709713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4052.534217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4053.352989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4054.412386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4055.236586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4056.239633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4057.118741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4058.162378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4059.023396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4059.887328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4060.735663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4061.605550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4062.473791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4063.438506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4064.368141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4065.250763] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4065.258333] Lustre: Skipped 257 previous similar messages [ 4065.276106] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4065.294088] Lustre: Skipped 257 previous similar messages [ 4065.298453] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4065.303987] Lustre: Skipped 257 previous similar messages [ 4065.308749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4065.322464] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4065.338482] Lustre: Skipped 257 previous similar messages [ 4066.267972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4067.183080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4068.179756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4068.987899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4069.890350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4070.888645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4071.799320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4072.782189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4073.900285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4074.779937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4075.861071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4076.894792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4077.806917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4078.770135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4079.619934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4080.418572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4081.123797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4081.851474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4082.664858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4083.433683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4084.305319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4085.269669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4086.180537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4087.094965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4088.042957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4088.895566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4089.784202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4090.586839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4091.484201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4092.360915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4093.248532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4094.298338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4095.141637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4095.839891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4096.743838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4097.599697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4098.664960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4099.513442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4100.407356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4101.364823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4102.494269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4103.301842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4104.073832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4104.829696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4105.612916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4106.438630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4107.301849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4108.182619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4109.004566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4109.900969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4110.779917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4111.694580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4112.552946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4113.533136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4114.313544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4115.449772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4116.497768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4117.564571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4118.398094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4119.485338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4120.391127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4121.225977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4121.974833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4122.908712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4123.917558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4124.970318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4125.991626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4126.971040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4127.969665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4128.929685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4129.778744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4130.705165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4131.641488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4132.495544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4133.348926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4134.271284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4135.120385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4136.277956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4137.200583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4138.234476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4139.201553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4140.191401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4141.039844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4141.966675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4142.747163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4143.699741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4144.527266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4145.402252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4146.198072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4147.031465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4147.904044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4148.893578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4149.578940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4150.455699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4151.286807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4152.054964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4152.879546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4153.739747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4154.691866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4155.576831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4156.586465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4157.410224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4158.337372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4159.314673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4160.248129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4161.173667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4162.089607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4162.796500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4163.747991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4164.804841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4165.625172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4166.610121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4167.476613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4168.334280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4169.327592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4170.317550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4171.102493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4171.981399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4172.793656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4173.564595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4174.286371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4175.275732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4176.164732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4177.079706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4177.909736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4178.902953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4179.793348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4180.784160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4181.596723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4182.493941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4183.189587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4184.130308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4185.025959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4185.876402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4186.690223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4187.656356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4188.372836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4189.413097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4190.200746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4191.125117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4192.004446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4193.151688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4194.041584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4194.913852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4195.797945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4196.757152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4197.635534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4198.510449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4199.487825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4200.322923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4201.126749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4202.162532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4203.003093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4203.833989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4204.631032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4205.673778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4206.743983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4207.739452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4208.803926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4209.755311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4210.669765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4211.451117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4212.292723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4213.229399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4214.091817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4214.839443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4215.674915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4216.489074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4217.381048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4218.297280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4219.185045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4219.956497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4220.734356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4221.474103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4222.204418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4222.960487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4223.727803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4224.507477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4225.321368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4226.112968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4226.938655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4227.781071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4228.552321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4229.425862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4230.198275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4230.922070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4231.721916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4232.555060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4233.338983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4234.076176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4234.760518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4235.414670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4236.131540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4236.823474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4237.587955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4238.285237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4239.105977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4239.937966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4240.759573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4241.640081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4242.562619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4243.553362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4244.643287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4245.448846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4246.430800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4247.352460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4248.236856] LustreError: 171853:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4248.245422] LustreError: 171853:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4248.254664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4249.052716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4249.768065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4250.570166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4251.351729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4252.242936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4253.173260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4254.006502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4254.898326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4255.866554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4256.734728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4257.509958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4258.459123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4259.233037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4260.072160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4260.833070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4261.742952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4262.655440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4263.635612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4264.511686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4265.654603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4266.529545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4267.415315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4268.227704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4269.035652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4269.910197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4270.769315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4271.637786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4272.439424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4273.261865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4274.062030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4274.918737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4275.750645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4276.584324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4277.373918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4278.164443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4278.797629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4279.596890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4280.311846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4281.121151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4281.827959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4282.620918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4283.421926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4284.299416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4285.048989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4285.966639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4286.965179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4287.977340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4288.904747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4289.863761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4290.766902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4291.714182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4292.494239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4293.302307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4294.039928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4294.930183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4295.715584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4296.516369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4297.382784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4298.260048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4299.081768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4299.862488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4300.683715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4301.621894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4302.510291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4303.355480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4304.368060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4305.301260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4306.285612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4307.244638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4308.301672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4309.195514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4310.041399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4310.792644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4311.582448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4312.445219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4313.341663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4314.112159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4314.865379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4315.674146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4316.487599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4317.288316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4318.124568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4319.005332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4319.875537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4320.668131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4321.653793] LustreError: 171853:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4321.671360] LustreError: 171853:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4321.673402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4322.577334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4323.520327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4324.351894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4325.142315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4325.990733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4326.799975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4327.588756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4328.501413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4329.263371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4330.086798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4330.837680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4331.773038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4332.616155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4333.419681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4334.147288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4334.917088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4335.603387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4336.529819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4337.432041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4338.278674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4339.113395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4340.155846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4341.059878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4341.970153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4342.699591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4343.450334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4344.155907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4344.933935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4345.877362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4346.670385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4347.513571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4348.312686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4349.101958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4349.929917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4350.711800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4351.422902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4352.177242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4352.966795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4353.841220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4354.695672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4355.465754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4356.522787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4357.285553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4358.178990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4359.009404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4359.777475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4360.702796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4361.640525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4362.568434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4363.311218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4364.007716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4364.867390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4365.669844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4366.536656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4367.278107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4368.076786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4368.879025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4369.640495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4370.504832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4371.321916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4372.198726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4373.077499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4373.828376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4374.700540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4375.459870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4376.284364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4377.105975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4377.885838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4378.698275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4379.791881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4380.661508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4381.482191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4382.355709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4383.409721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4384.392143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4385.346070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4386.260069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4387.359350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4388.198751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4389.198854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4390.152846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4391.349478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4392.304797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4393.260232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4394.191517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4395.175766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4396.082551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4397.123456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4398.094339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4399.164185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4400.292512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4401.484966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4402.573466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4403.873270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4404.928020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4405.781462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4406.850628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4407.805581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4408.703358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4409.818275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4410.696952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4411.498522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4412.269846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4413.170950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4413.952574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4414.997745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4415.870301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4416.672703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4417.534905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4418.363316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4419.206712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4420.012640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4420.859635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4421.745544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4422.519730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4423.383959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4424.213494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4424.963516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4425.794279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4426.831859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4427.665727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4428.487385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4429.288745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4430.147104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4430.952863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4431.763852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4432.457122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4433.178793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4433.865236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4434.570913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4435.291188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4436.085180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4436.911518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4437.741466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4438.479814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4439.446550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4440.433536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4441.325788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4442.213554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4442.976285] LustreError: 171853:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4442.984941] LustreError: 171853:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4442.997940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4443.759976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4444.624810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4445.339161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4446.065295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4446.757036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4447.523346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4448.420535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4449.159155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4449.914565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4450.704621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4451.559817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4452.383446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4453.270889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4454.060024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4454.727381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4455.435121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4456.113730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4456.875804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4802.050609] LustreError: 174042:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 4802.065074] LustreError: 171853:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 5272.545250] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5272.553618] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5272.564950] Lustre: Skipped 451 previous similar messages [ 5272.572555] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5274.597871] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5274.615090] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5275.617321] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5275.633639] Lustre: Skipped 1 previous similar message [ 5277.568223] Lustre: server umount lustre-OST0000 complete [ 5285.346504] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5285.366596] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5287.789972] Lustre: server umount lustre-MDT0000 complete [ 5290.978773] LustreError: 171834:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5291.010518] LustreError: 171834:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 5292.849425] LustreError: 171813:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788274486 with bad export cookie 17913525657174007803 [ 5292.865038] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5292.875611] LustreError: 171813:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5293.687949] Lustre: server umount lustre-MDT0001 complete [ 5307.882951] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 5311.010933] Key type lgssc unregistered [ 5311.321353] LNet: 227062:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5311.325164] LNetError: 227062:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5311.336815] LNet: Removed LNI 192.168.203.107@tcp [ 5312.120863] Key type .llcrypt unregistered [ 5312.125118] Key type ._llcrypt unregistered [ 5335.485262] Key type ._llcrypt registered [ 5335.493252] Key type .llcrypt registered [ 5335.635580] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 5336.477066] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5336.486468] alg: No test for adler32 (adler32-zlib) [ 5337.632706] Lustre: Lustre: Build Version: 2.17.57_82_g90dbd01 [ 5337.953075] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 5339.631260] Key type lgssc registered [ 5341.025257] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5349.015864] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5349.050632] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5350.434834] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5354.920424] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 5360.616865] LustreError: 228479:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5360.660960] LustreError: 228479:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5365.016729] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5365.585308] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5370.468850] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 5379.609866] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5385.776866] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5392.104858] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5392.550869] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5395.643061] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 5400.531290] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 5410.360469] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5415.793697] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5417.921320] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5417.954944] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5417.969630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.992323] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5417.992933] LustreError: 228498:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 5421.547698] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5421.551678] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5421.596293] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5422.644543] Lustre: lustre-OST0000: Not available for connect from 192.168.203.7@tcp (stopping) [ 5422.652995] Lustre: Skipped 1 previous similar message [ 5426.572824] Lustre: server umount lustre-OST0000 complete [ 5438.064213] LustreError: 228498:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 5460.366165] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5460.698255] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5461.738639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.747238] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.749596] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5461.768062] Lustre: Skipped 1 previous similar message [ 5465.738926] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 5472.633600] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5474.139351] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 5479.877045] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8c8986b8b000.ost_server_uuid 50 [ 5481.413057] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8c8986b8b000.ost_server_uuid in FULL state after 0 sec [ 5487.072217] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5487.078783] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5487.091884] Lustre: Skipped 1 previous similar message [ 5487.098730] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5487.615739] Lustre: server umount lustre-OST0000 complete [ 5492.712157] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5492.715834] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5492.728808] Lustre: Skipped 1 previous similar message [ 5496.644714] Lustre: server umount lustre-MDT0000 complete [ 5497.827706] LustreError: 230187:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5497.865608] LustreError: 230187:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 5499.990656] LustreError: 228459:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788274693 with bad export cookie 3321327567864227742 [ 5499.992035] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5500.002500] LustreError: 228459:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5509.831691] Lustre: server umount lustre-MDT0001 complete [ 5523.087387] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 5526.280982] Key type lgssc unregistered [ 5526.626856] LNet: 233692:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5526.635914] LNetError: 233692:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5527.659243] LNet: Removed LNI 192.168.203.107@tcp [ 5528.418827] Key type .llcrypt unregistered [ 5528.423593] Key type ._llcrypt unregistered [ 5559.167825] Key type ._llcrypt registered [ 5559.171052] Key type .llcrypt registered [ 5559.284887] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 5559.806489] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5559.862063] alg: No test for adler32 (adler32-zlib) [ 5561.017355] Lustre: Lustre: Build Version: 2.17.57_82_g90dbd01 [ 5561.267717] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 5562.920304] Key type lgssc registered [ 5563.834635] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5572.892183] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5572.916303] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5574.538349] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5579.859972] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 5584.872327] LustreError: 235416:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5584.900675] LustreError: 235416:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5589.643851] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5589.985921] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 5590.425298] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5598.514383] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 5601.898450] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 5609.488157] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5609.957800] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5617.445389] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 5619.196751] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 5627.064761] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5627.322358] Lustre: lustre-OST0001: new disk, initializing [ 5627.329986] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5627.344624] Lustre: 237728:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5627.460970] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5633.609677] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 5634.714095] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5634.734595] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5634.915675] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5645.708920] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5653.997342] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 5670.886378] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5670.887776] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5670.917943] Lustre: Skipped 2 previous similar messages [ 5670.936593] Lustre: Skipped 3 previous similar messages [ 5673.963681] Lustre: server umount lustre-MDT0000 complete [ 5676.005232] LustreError: 235413:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5676.021435] LustreError: 235413:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 5677.290160] LustreError: 235399:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788274871 with bad export cookie 6200439418536230023 [ 5677.300639] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5677.303819] LustreError: 235399:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5677.644484] Lustre: server umount lustre-MDT0001 complete [ 5691.676336] Lustre: server umount lustre-OST0000 complete [ 5705.943929] Lustre: server umount lustre-OST0001 complete [ 5729.008361] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 5737.967647] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5738.514164] LustreError: 240845:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5738.612759] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5742.430980] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 5744.100172] LustreError: 240846:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5748.194827] LustreError: 240845:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5750.108704] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5750.346710] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5754.618235] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 5757.726829] Lustre: 241988:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5765.062506] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5773.118598] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 5775.799172] LustreError: 242340:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5775.831334] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [ 5784.050441] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 5784.392061] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5784.629590] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5784.641164] Lustre: Skipped 1 previous similar message [ 5792.766394] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 5802.077532] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5805.660559] Lustre: 243857:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5816.809703] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5816.826147] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5816.847534] Lustre: Skipped 1 previous similar message [ 5816.876340] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5820.396530] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5820.396662] Lustre: mylustre-MDT0000-lwp-OST0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5820.418849] Lustre: Skipped 2 previous similar messages [ 5822.784240] Lustre: server umount mylustre-MDT0000 complete [ 5825.545134] LustreError: 241579:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5825.616457] LustreError: 241579:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 5828.086483] LustreError: 240822:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788275021 with bad export cookie 6200439418536232886 [ 5828.104639] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5828.506418] Lustre: server umount mylustre-MDT0001 complete [ 5844.671964] Lustre: server umount mylustre-OST0000 complete [ 5847.007146] Lustre: 234681:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788275024/real 1788275024] req@ffff9bc2128c0700 x1875141989661824/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788275040 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 5847.051085] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5848.321254] Lustre: server umount mylustre-OST0001 complete [ 5870.263795] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 5881.784194] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5882.087644] LustreError: 246168:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5882.157179] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5887.100136] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 5896.584790] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5903.025029] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 5906.184401] Lustre: 247309:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5913.328319] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5921.873680] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 5921.969641] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:161) [ 5927.104368] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 5930.073170] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5935.618565] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:65) [ 5936.413892] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 5943.963467] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5947.333754] Lustre: 249181:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5956.066163] Lustre: tfs-MDT0000-lwp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5956.074933] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 5956.088036] Lustre: Skipped 3 previous similar messages [ 5956.113542] Lustre: Skipped 6 previous similar messages [ 5960.282613] Lustre: server umount tfs-MDT0000 complete [ 5961.191687] LustreError: 246169:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5961.224640] LustreError: 246169:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [ 5964.535982] LustreError: 246143:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788275158 with bad export cookie 6200439418536234839 [ 5964.551122] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5964.560165] LustreError: 246143:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 5965.015497] Lustre: server umount tfs-MDT0001 complete [ 5979.646839] Lustre: server umount tfs-OST0000 complete [ 5982.688075] Lustre: 234681:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788275160/real 1788275160] req@ffff9bc32726bb80 x1875141989713536/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788275176 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 5982.710200] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5984.247844] Lustre: server umount tfs-OST0001 complete [ 6005.339595] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 6014.851936] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6015.309497] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6015.315414] Lustre: Skipped 3 previous similar messages [ 6019.913768] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 6026.896713] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6031.201852] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 6034.081893] Lustre: 252577:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6040.085665] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6041.455635] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 6047.000938] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 6051.845150] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:193) [ 6056.652733] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6062.069857] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:97) [ 6063.925859] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 6072.837428] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6076.547251] Lustre: 254445:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6083.633487] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 11:07:56 (1788275276) [ 6103.012584] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6103.028051] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6103.040033] Lustre: Skipped 2 previous similar messages [ 6103.062634] Lustre: Skipped 3 previous similar messages [ 6106.628088] Lustre: server umount lustre-MDT0000 complete [ 6108.133331] LustreError: 251433:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6108.154446] LustreError: 251433:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 10 previous similar messages [ 6110.309216] LustreError: 251409:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788275304 with bad export cookie 6200439418536236757 [ 6110.318048] LustreError: 251409:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6110.318271] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6116.759672] Lustre: server umount lustre-MDT0001 complete [ 6126.660233] Lustre: server umount lustre-OST0000 complete [ 6136.392375] Lustre: server umount lustre-OST0001 complete [ 6141.471157] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [ 6150.418799] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 6187.521827] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6187.766861] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6187.792949] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6187.890645] Lustre: lustre-MDT0000: new disk, initializing [ 6187.970658] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 6187.977676] Lustre: Skipped 3 previous similar messages [ 6187.996096] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6192.386781] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 6202.825509] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6202.926498] Lustre: 257630:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6202.971438] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6202.973480] Lustre: Skipped 1 previous similar message [ 6203.020966] Lustre: lustre-MDT0001: new disk, initializing [ 6203.201830] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6203.210584] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6208.028797] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 6220.444633] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6220.957978] Lustre: lustre-OST0000: new disk, initializing [ 6220.963980] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6220.973809] Lustre: 259213:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6222.061592] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6222.089490] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6222.293199] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6228.854929] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 6240.017175] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6240.248980] Lustre: lustre-OST0001: new disk, initializing [ 6240.256806] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 6240.275775] Lustre: 260085:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6242.279284] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 6242.320220] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 6242.472343] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 6247.150984] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 6260.721294] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 11:10:53 (1788275453) [ 6280.358982] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 11:11:12 (1788275472) [ 6298.595994] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6298.607419] Lustre: Skipped 3 previous similar messages [ 6298.619899] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6298.625836] Lustre: Skipped 5 previous similar messages [ 6310.368662] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6310.575858] Lustre: server umount lustre-OST0000 complete [ 6323.402245] Lustre: server umount lustre-MDT0000 complete [ 6329.215790] LustreError: 257623:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788275523 with bad export cookie 6200439418536238535 [ 6329.223427] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6329.261255] LustreError: 257623:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6334.435350] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6334.441775] Lustre: Skipped 8 previous similar messages [ 6335.648725] Lustre: server umount lustre-MDT0001 complete [ 6344.764319] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 6418.381587] Lustre: server umount lustre-OST0001 complete [ 6424.897504] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [ 6441.665368] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 6482.125283] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6482.517607] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6482.565402] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6482.672809] Lustre: lustre-MDT0000: new disk, initializing [ 6482.836533] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6482.843320] Lustre: Skipped 3 previous similar messages [ 6482.898215] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6489.101582] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 6500.248678] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6500.319109] Lustre: 264950:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6500.373557] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6500.381760] Lustre: Skipped 1 previous similar message [ 6500.432504] Lustre: lustre-MDT0001: new disk, initializing [ 6500.513229] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6500.521905] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6504.603926] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 6514.374520] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6520.581781] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6528.993957] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6529.271571] Lustre: 266917:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6530.661579] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6530.789289] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6536.360555] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 6547.021528] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6556.640949] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6556.647950] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6556.654819] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6556.677802] Lustre: Skipped 5 previous similar messages [ 6562.320039] Lustre: server umount lustre-OST0000 complete [ 6567.392441] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6573.569395] Lustre: server umount lustre-MDT0000 complete [ 6577.127531] LustreError: 265749:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6577.168030] LustreError: 265749:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 14 previous similar messages [ 6578.910738] LustreError: 264941:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788275772 with bad export cookie 6200439418536240005 [ 6578.932806] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6578.939502] LustreError: 264941:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6579.266261] Lustre: server umount lustre-MDT0001 complete [ 6590.808207] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 6593.708349] Key type lgssc unregistered [ 6593.980067] LNet: 268682:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6593.985378] LNetError: 268682:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6595.045673] LNet: Removed LNI 192.168.203.107@tcp [ 6595.964314] Key type .llcrypt unregistered [ 6595.967198] Key type ._llcrypt unregistered [ 6626.625835] Key type ._llcrypt registered [ 6626.627945] Key type .llcrypt registered [ 6626.701972] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 6627.958975] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6627.982601] alg: No test for adler32 (adler32-zlib) [ 6629.046697] Lustre: Lustre: Build Version: 2.17.57_82_g90dbd01 [ 6629.274263] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 6630.927268] Key type lgssc registered [ 6631.830635] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6638.945931] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6638.977364] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6640.496352] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6644.162975] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 6650.865378] LustreError: 270145:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6650.904245] LustreError: 270145:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 6652.794443] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6653.362550] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6658.127188] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 6668.508247] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6677.316840] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6686.094681] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6686.649905] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6693.114418] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 6700.840276] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6702.224961] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 6707.185325] Lustre: server umount lustre-OST0000 complete [ 6714.851320] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6714.862099] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6714.876023] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6716.387488] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6716.387504] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6717.648854] Lustre: server umount lustre-MDT0000 complete [ 6721.507045] LustreError: 270145:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6721.527328] LustreError: 270145:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6721.569065] LustreError: 270170:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788275915 with bad export cookie 14752298168122636017 [ 6721.570328] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6721.578390] LustreError: 270170:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6731.982108] Lustre: server umount lustre-MDT0001 complete [ 6745.391500] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 6749.047337] Key type lgssc unregistered [ 6749.410726] LNet: 273734:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6749.421745] LNetError: 273734:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6749.434865] LNet: Removed LNI 192.168.203.107@tcp [ 6750.446635] Key type .llcrypt unregistered [ 6750.452866] Key type ._llcrypt unregistered [ 6766.256976] Key type ._llcrypt registered [ 6766.259507] Key type .llcrypt registered [ 6766.399095] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 6767.628677] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6767.710455] alg: No test for adler32 (adler32-zlib) [ 6768.744382] Lustre: Lustre: Build Version: 2.17.57_82_g90dbd01 [ 6769.139683] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 6770.799357] Key type lgssc registered [ 6771.968789] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6782.025101] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6782.063034] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6797.663501] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 6802.783427] LustreError: 274929:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.107@tcp: failed processing log, type 4: rc = -110 [ 6829.535295] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6835.412903] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 6843.425825] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6845.653517] Lustre: server umount lustre-OST0000 complete [ 6863.851540] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 6866.877644] Key type lgssc unregistered [ 6867.201657] LNet: 276563:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6867.218175] LNetError: 276563:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6868.262261] LNet: Removed LNI 192.168.203.107@tcp [ 6869.226107] Key type .llcrypt unregistered [ 6869.231474] Key type ._llcrypt unregistered [ 6911.242623] Key type ._llcrypt registered [ 6911.244427] Key type .llcrypt registered [ 6911.349829] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 6912.438910] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6912.595658] alg: No test for adler32 (adler32-zlib) [ 6913.672500] Lustre: Lustre: Build Version: 2.17.57_82_g90dbd01 [ 6913.935636] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 6915.607361] Key type lgssc registered [ 6917.001888] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6931.031305] loop0: detected capacity change from 0 to 204800000 [ 6931.246581] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6931.265607] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6931.459548] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6932.845810] loop0: detected capacity change from 0 to 204800000 [ 6933.137222] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6933.156450] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6933.799996] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6935.135456] loop0: detected capacity change from 0 to 204800000 [ 6935.404986] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6935.425850] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6935.613309] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6937.102691] loop0: detected capacity change from 0 to 204800000 [ 6937.386036] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6937.392868] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6937.486839] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6938.996888] loop0: detected capacity change from 0 to 204800000 [ 6939.061275] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6944.181384] loop0: detected capacity change from 0 to 204800000 [ 6944.205768] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6949.583614] loop0: detected capacity change from 0 to 204800000 [ 6949.620421] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6954.280706] loop0: detected capacity change from 0 to 204800000 [ 6954.338139] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6958.498437] loop0: detected capacity change from 0 to 204800000 [ 6958.600139] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6958.670146] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6958.765144] Lustre: 279979:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6958.785210] Lustre: 279979:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 6958.805544] Lustre: 279979:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 6958.821206] Lustre: 279979:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 6958.846745] Lustre: 279979:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 6958.856711] Lustre: 279979:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 6958.872642] Lustre: 279979:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 6958.889169] Lustre: 279979:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 6958.906754] Lustre: 279979:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 6958.922114] Lustre: 279979:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 6958.940261] Lustre: 279979:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 6958.964606] Lustre: 279979:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 6958.980013] Lustre: 279979:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 6958.995292] Lustre: 279979:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 6964.551474] Lustre: server umount MGS complete [ 6965.575653] loop0: detected capacity change from 0 to 204800000 [ 6965.739687] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6966.201308] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6966.254599] Lustre: lustre-MDT0000: Aborting client recovery [ 6967.100362] loop1: detected capacity change from 0 to 204800000 [ 6967.315907] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6967.440203] Lustre: 280409:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6967.453735] Lustre: 280409:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 6967.462850] Lustre: 280409:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 6967.470879] Lustre: 280409:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 6967.479954] Lustre: 280409:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 6967.487970] Lustre: 280409:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 6967.495831] Lustre: 280409:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 6967.516904] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6967.687103] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6967.743243] Lustre: lustre-MDT0001: Aborting client recovery [ 6968.753600] loop2: detected capacity change from 0 to 204800000 [ 6968.945882] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6969.022439] Lustre: 280489:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6969.031605] Lustre: 280489:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 6969.043800] Lustre: 280489:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 6969.063207] Lustre: 280489:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 6969.079271] Lustre: 280489:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 6969.091235] Lustre: 280489:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 6969.101977] Lustre: 280489:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 6969.260699] Lustre: Found index 0 for lustre-OST0000, updating log [ 6969.325676] Lustre: 280508:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6969.409561] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6969.432297] Lustre: lustre-OST0000: Aborting recovery [ 6970.488760] loop3: detected capacity change from 0 to 204800000 [ 6970.626094] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6970.720881] Lustre: 280574:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6970.728753] Lustre: 280574:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 6970.737096] Lustre: 280574:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 6970.751759] Lustre: 280574:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 6970.769512] Lustre: 280574:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 6970.786528] Lustre: 280574:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 6970.800469] Lustre: 280574:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 6970.816304] Lustre: Found index 1 for lustre-OST0001, updating log [ 6970.846313] Lustre: 280579:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6970.922481] Lustre: lustre-OST0001: Aborting recovery [ 6976.016423] LustreError: 280353:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 9, retries 0, failed: rc = -108 [ 6977.035588] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 6977.049385] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 6977.088525] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 6977.109422] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 6978.087912] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 6978.089280] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 6980.164541] LustreError: 280320:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 5651us: rc = -2 [ 6980.680221] LustreError: 280320:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 515us: rc = -2 [ 6980.695550] LustreError: 280320:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 17 previous similar messages [ 6998.500314] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6998.504410] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6998.506718] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6998.525121] Lustre: Skipped 2 previous similar messages [ 7000.104248] Lustre: server umount lustre-MDT0000 complete [ 7001.015918] LustreError: 280305:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788276194 with bad export cookie 6166359601826397526 [ 7001.016733] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7001.024977] LustreError: 280305:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7003.621877] LustreError: 280319:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7003.622725] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7003.626968] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7003.626979] Lustre: Skipped 3 previous similar messages [ 7003.646421] LustreError: 280319:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 7003.654127] Lustre: Skipped 2 previous similar messages [ 7006.625195] LustreError: 280517:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7006.628051] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7006.656875] Lustre: Skipped 2 previous similar messages [ 7010.723993] LustreError: 280319:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7010.724312] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7010.745084] LustreError: 280319:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7010.765365] Lustre: Skipped 1 previous similar message [ 7015.393099] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7015.797723] Lustre: server umount lustre-MDT0001 complete [ 7031.264526] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7031.492313] Lustre: server umount lustre-OST0000 complete [ 7038.744473] Lustre: server umount lustre-OST0001 complete [ 7046.691624] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 11:23:59 (1788276239) [ 7077.186095] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [ 7086.490245] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 7121.070319] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7121.309281] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7121.335894] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7121.424915] Lustre: lustre-MDT0000: new disk, initializing [ 7121.477892] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7121.482837] Lustre: Skipped 1 previous similar message [ 7121.493510] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7125.810315] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 7136.515105] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7136.614176] Lustre: 284649:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7136.665349] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7136.673320] Lustre: Skipped 1 previous similar message [ 7136.775956] Lustre: lustre-MDT0001: new disk, initializing [ 7136.902532] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7136.935113] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7136.956642] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7140.895298] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 7149.772816] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7156.483764] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7164.034636] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7164.331535] Lustre: lustre-OST0000: new disk, initializing [ 7164.335756] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7164.348529] Lustre: 286616:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7164.452680] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7171.808179] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 7174.744602] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7174.755919] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7174.881466] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7181.865738] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7187.787668] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 7202.192660] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 7208.755343] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 7262.177459] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7262.195845] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7262.214609] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7262.227316] Lustre: Skipped 1 previous similar message [ 7271.392899] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7271.677111] Lustre: server umount lustre-OST0000 complete [ 7277.539826] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7277.556404] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7277.556941] Lustre: Skipped 2 previous similar messages [ 7277.572814] Lustre: Skipped 3 previous similar messages [ 7281.764766] Lustre: server umount lustre-MDT0000 complete [ 7282.666530] LustreError: 285811:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7282.698144] LustreError: 285811:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7285.307616] LustreError: 287476:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788276479 with bad export cookie 6166359601826400242 [ 7285.311161] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7285.317347] LustreError: 287476:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7285.778799] Lustre: server umount lustre-MDT0001 complete [ 7294.478519] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7299.164160] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 7304.224594] Lustre: server umount MGS complete [ 7316.062782] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7316.639129] LustreError: 290534:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7316.686482] LustreError: 290547:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 7316.790436] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7321.745764] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 7326.188410] LustreError: 290535:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7326.235538] LustreError: 290535:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7331.111359] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7331.316407] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 7336.105468] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 7344.177645] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7350.026714] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7356.283767] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7356.391775] LustreError: 292356:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7356.627772] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7356.639950] Lustre: Skipped 1 previous similar message [ 7362.856126] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 7370.676682] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7402.986340] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7402.987543] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7403.009605] LustreError: Skipped 2 previous similar messages [ 7403.012955] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7403.019918] Lustre: Skipped 1 previous similar message [ 7406.616529] Lustre: server umount lustre-OST0000 complete [ 7413.216855] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7416.910139] Lustre: server umount lustre-MDT0000 complete [ 7418.340732] LustreError: 290529:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7418.352559] LustreError: 290529:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7420.218203] LustreError: 293260:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788276614 with bad export cookie 6166359601826402076 [ 7420.227895] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7420.240182] LustreError: 293260:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7420.879140] Lustre: server umount lustre-MDT0001 complete [ 7431.411899] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 7434.726945] Key type lgssc unregistered [ 7435.066334] LNet: 294483:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7435.076694] LNetError: 294483:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7435.098789] LNet: Removed LNI 192.168.203.107@tcp [ 7436.298670] Key type .llcrypt unregistered [ 7436.300445] Key type ._llcrypt unregistered [ 7478.779127] Key type ._llcrypt registered [ 7478.789156] Key type .llcrypt registered [ 7478.909321] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [ 7495.064575] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 7496.327033] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7496.421212] alg: No test for adler32 (adler32-zlib) [ 7497.528854] Lustre: Lustre: Build Version: 2.17.57_82_g90dbd01 [ 7497.794495] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 7499.479160] Key type lgssc registered [ 7500.509496] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7536.404847] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7536.453295] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7537.764682] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7537.799382] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7537.880322] Lustre: lustre-MDT0000: new disk, initializing [ 7538.015722] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7538.043513] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7542.456059] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 7553.109756] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7553.239988] Lustre: 298456:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7553.300946] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7553.304511] Lustre: Skipped 1 previous similar message [ 7553.360626] Lustre: lustre-MDT0001: new disk, initializing [ 7553.444942] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7553.497292] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7553.511801] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7558.321722] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 7568.962221] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7575.419736] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7583.584467] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7583.818667] Lustre: lustre-OST0000: new disk, initializing [ 7583.822492] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7583.832603] Lustre: 300423:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7583.958671] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7589.434239] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7589.451436] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7589.613701] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7590.508313] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 7600.515719] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7609.001158] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 7623.002051] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 7631.423141] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 7676.900055] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7676.905165] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7676.930490] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7676.936699] LustreError: Skipped 1 previous similar message [ 7676.954092] Lustre: Skipped 1 previous similar message [ 7682.021544] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7682.037629] Lustre: Skipped 1 previous similar message [ 7682.411979] Lustre: server umount lustre-OST0000 complete [ 7686.627954] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7686.633217] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7686.641166] Lustre: Skipped 1 previous similar message [ 7686.652986] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7692.365565] Lustre: server umount lustre-MDT0000 complete [ 7696.177524] LustreError: 298447:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788276890 with bad export cookie 3491259378435526165 [ 7696.187571] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7696.195161] LustreError: 298447:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7696.635946] Lustre: server umount lustre-MDT0001 complete [ 7705.097822] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7709.829956] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 7715.200944] Lustre: server umount MGS complete [ 7727.065268] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7727.516223] LustreError: 304377:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7727.534254] LustreError: 304377:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 7727.547449] LustreError: 304390:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 7727.626794] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7732.579410] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 7732.712241] LustreError: 304378:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7737.825636] LustreError: 304377:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7741.818598] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7742.182302] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7747.099286] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 7754.724724] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7761.078473] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7768.513674] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7768.927350] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7775.111599] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 7783.149252] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7817.194431] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7817.198921] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7817.213737] Lustre: Skipped 2 previous similar messages [ 7817.216380] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7817.216391] Lustre: Skipped 2 previous similar messages [ 7822.401314] Lustre: server umount lustre-OST0000 complete [ 7829.472535] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7829.486546] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7829.505643] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7829.519867] Lustre: Skipped 1 previous similar message [ 7837.664593] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7837.670355] Lustre: Skipped 3 previous similar messages [ 7842.273874] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7842.473952] Lustre: server umount lustre-MDT0000 complete [ 7842.797511] LustreError: 304373:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7842.830088] LustreError: 304373:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7846.845546] LustreError: 304357:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788277040 with bad export cookie 3491259378435527894 [ 7846.849433] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7846.861102] LustreError: 304357:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7847.082877] Lustre: server umount lustre-MDT0001 complete [ 7858.478642] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 7861.227960] Key type lgssc unregistered [ 7861.507599] LNet: 308329:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7861.513207] LNetError: 308329:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7861.527743] LNet: Removed LNI 192.168.203.107@tcp [ 7862.462145] Key type .llcrypt unregistered [ 7862.466515] Key type ._llcrypt unregistered [ 7883.712751] Key type ._llcrypt registered [ 7883.717752] Key type .llcrypt registered [ 7883.847321] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [ 7897.631892] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 7898.606470] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7898.706590] alg: No test for adler32 (adler32-zlib) [ 7899.783635] Lustre: Lustre: Build Version: 2.17.57_82_g90dbd01 [ 7900.026971] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 7901.711171] Key type lgssc registered [ 7902.884397] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7926.786636] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 7940.647566] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7940.682061] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7941.918911] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7941.962557] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7942.018596] Lustre: lustre-MDT0000: new disk, initializing [ 7942.081037] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7942.120591] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7946.437759] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 7957.885941] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7958.226798] Lustre: lustre-OST0000: new disk, initializing [ 7958.231517] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7958.240052] Lustre: Skipped 1 previous similar message [ 7958.245149] Lustre: 312379:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7958.346397] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7966.146948] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 7968.382715] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 7968.403202] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 7968.472443] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 8928.740152] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8928.758398] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8933.867066] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8934.269659] Lustre: server umount lustre-MDT0000 complete [ 8938.465581] LustreError: 311565:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788278132 with bad export cookie 13025430330303031315 [ 8938.467559] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8938.484572] LustreError: 311565:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8938.722355] Lustre: server umount lustre-OST0000 complete [ 8964.155697] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [ 8973.877959] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 9018.593489] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9018.842210] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9018.867855] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9018.937631] Lustre: lustre-MDT0000: new disk, initializing [ 9019.035724] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9019.054992] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9023.807813] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9035.453175] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9035.609955] Lustre: 316734:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9035.700545] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9035.711779] Lustre: Skipped 1 previous similar message [ 9035.840476] Lustre: lustre-MDT0001: new disk, initializing [ 9036.014032] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9036.105045] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9036.129909] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9041.098191] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9052.814750] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9060.365952] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9068.524787] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9068.798796] Lustre: lustre-OST0000: new disk, initializing [ 9068.802112] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9068.812966] Lustre: 318702:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9068.933860] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9075.279040] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9075.288834] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9075.432502] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9075.912474] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9086.598187] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9093.281475] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9093.661379] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9098.972990] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 9099.244423] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9106.401983] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9106.403989] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9106.408503] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9106.415202] LustreError: Skipped 1 previous similar message [ 9106.422762] Lustre: Skipped 1 previous similar message [ 9107.597258] Lustre: server umount lustre-OST0000 complete [ 9111.524284] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9111.526263] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9111.537943] Lustre: Skipped 1 previous similar message [ 9111.549744] Lustre: Skipped 2 previous similar messages [ 9116.641034] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9116.646972] Lustre: Skipped 1 previous similar message [ 9117.809872] Lustre: server umount lustre-MDT0000 complete [ 9122.197033] LustreError: 316728:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788278316 with bad export cookie 13025430330305972141 [ 9122.209816] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9122.211117] LustreError: 316728:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9122.813651] Lustre: server umount lustre-MDT0001 complete [ 9127.057477] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 9128.962649] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 11:58:41 (1788278321) [ 9136.177577] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9136.744579] LustreError: 321014:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9136.775864] LustreError: 321014:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9136.866542] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9141.657732] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9142.240535] LustreError: 321015:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9146.338996] LustreError: 321014:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9150.629043] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9151.174596] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9155.918280] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9163.723225] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9169.364318] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9175.726589] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9175.974930] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9182.235280] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9189.933616] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9197.488263] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9197.655842] Lustre: lustre-OST0001: new disk, initializing [ 9197.662044] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 9197.669616] Lustre: 323851:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9197.781330] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9203.834625] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9207.400389] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 9207.412363] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 9207.486030] LustreError: 324345:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9208.552419] LustreError: 324356:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9208.570676] LustreError: 324356:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 9209.583712] LustreError: 324345:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9211.681123] LustreError: 324345:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9211.697230] LustreError: 324345:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 4 previous similar messages [ 9213.610695] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9213.777606] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9215.781807] LustreError: 324356:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9215.791766] LustreError: 324356:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 6 previous similar messages [ 9219.600643] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8c89860ea000.ost_server_uuid 50 [ 9221.167880] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8c89860ea000.ost_server_uuid in FULL state after 0 sec [ 9224.063284] LustreError: 324345:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9224.078938] LustreError: 324345:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [ 9227.277654] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8c89860ea000.ost_server_uuid 50 [ 9228.892698] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8c89860ea000.ost_server_uuid in FULL state after 0 sec [ 9231.380495] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 9245.264406] Lustre: lustre-OST0001: Not available for connect from 192.168.203.7@tcp (stopping) [ 9246.697051] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9246.698243] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9246.709105] LustreError: Skipped 1 previous similar message [ 9250.347516] LustreError: 322839:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.203.7@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9250.461398] Lustre: server umount lustre-OST0001 complete [ 9258.981670] LustreError: 324347:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9258.988589] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9258.988836] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9258.988840] Lustre: Skipped 1 previous similar message [ 9259.009435] LustreError: 324347:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9259.060076] LustreError: Skipped 1 previous similar message [ 9262.904773] Lustre: server umount lustre-OST0000 complete [ 9269.216683] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9269.225178] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9269.237829] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9269.256896] Lustre: Skipped 2 previous similar messages [ 9269.270924] Lustre: Skipped 7 previous similar messages [ 9273.184655] Lustre: server umount lustre-MDT0000 complete [ 9274.349547] LustreError: 322027:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9274.372161] LustreError: 322027:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9278.165674] LustreError: 323850:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788278472 with bad export cookie 13025430330305973023 [ 9278.179750] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9278.181829] LustreError: 323850:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9278.769957] Lustre: server umount lustre-MDT0001 complete [ 9291.495258] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 9294.285624] Key type lgssc unregistered [ 9294.621438] LNet: 326554:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9294.631257] LNetError: 326554:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9294.647837] LNet: Removed LNI 192.168.203.107@tcp [ 9295.582318] Key type .llcrypt unregistered [ 9295.584427] Key type ._llcrypt unregistered [ 9322.613207] Key type ._llcrypt registered [ 9322.619831] Key type .llcrypt registered [ 9322.768601] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 9323.667189] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9323.811337] alg: No test for adler32 (adler32-zlib) [ 9324.943283] Lustre: Lustre: Build Version: 2.17.57_82_g90dbd01 [ 9325.221473] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 9326.935173] Key type lgssc registered [ 9328.297127] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9336.529712] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9336.552652] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9338.067422] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9342.579414] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9347.559449] LustreError: 327977:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9347.581757] LustreError: 327977:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 9350.520442] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9352.681937] LustreError: 327976:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9357.142772] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9357.453916] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9361.495952] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9370.136285] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9370.497279] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9375.931466] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9382.763892] LustreError: 330288:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9382.779785] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 9382.793423] LustreError: 330288:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9383.060317] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9388.003304] LustreError: 330290:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9388.038975] LustreError: 330290:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9389.802144] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9390.038469] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9395.197092] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 9396.590541] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9403.672257] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9410.174164] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 9410.367788] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9518.049945] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9518.068302] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9518.069524] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 9518.096272] Lustre: Skipped 1 previous similar message [ 9520.375045] Lustre: server umount lustre-OST0001 complete [ 9523.177301] LustreError: 330290:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9523.191641] LustreError: 330290:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9528.296974] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9528.299364] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9528.307120] Lustre: Skipped 1 previous similar message [ 9528.314963] Lustre: Skipped 1 previous similar message [ 9532.802139] Lustre: server umount lustre-OST0000 complete [ 9536.482870] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9536.487698] LustreError: Skipped 1 previous similar message [ 9536.490546] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9536.519560] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9538.534198] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9538.534790] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9542.675940] Lustre: server umount lustre-MDT0000 complete [ 9543.660901] LustreError: 327977:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9543.685133] LustreError: 327977:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9546.098498] LustreError: 331667:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788278739 with bad export cookie 334000118704263249 [ 9546.108284] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9546.117227] LustreError: 331667:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9546.388944] Lustre: server umount lustre-MDT0001 complete [ 9556.567604] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 9559.115122] Key type lgssc unregistered [ 9559.453314] LNet: 332924:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9559.460154] LNetError: 332924:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9559.493700] LNet: Removed LNI 192.168.203.107@tcp [ 9560.483246] Key type .llcrypt unregistered [ 9560.492719] Key type ._llcrypt unregistered [ 9611.196606] Key type ._llcrypt registered [ 9611.200746] Key type .llcrypt registered [ 9611.299156] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 9612.212251] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9612.263540] alg: No test for adler32 (adler32-zlib) [ 9613.451344] Lustre: Lustre: Build Version: 2.17.57_82_g90dbd01 [ 9613.685982] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 9615.352449] Key type lgssc registered [ 9616.550047] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9626.664619] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9626.687892] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9628.223071] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9632.189907] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9638.370696] LustreError: 335219:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9638.391714] LustreError: 335219:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 9640.133475] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9640.588838] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9645.253316] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9648.311105] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 9654.799697] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9655.216479] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9661.288747] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9666.478557] LustreError: 336709:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9670.375185] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9670.879468] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9673.986973] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 9673.988737] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 9677.207105] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9686.119578] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9692.080136] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 9706.977659] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9706.985544] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9707.041119] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9710.058345] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9710.064852] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9710.078551] Lustre: Skipped 1 previous similar message [ 9710.101757] Lustre: Skipped 1 previous similar message [ 9712.042426] Lustre: server umount lustre-MDT0000 complete [ 9715.178060] LustreError: 335948:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9715.206174] LustreError: 335948:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9715.818471] LustreError: 335199:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788278909 with bad export cookie 1100603521274637805 [ 9715.846794] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9716.141269] Lustre: server umount lustre-MDT0001 complete [ 9730.683602] Lustre: server umount lustre-OST0000 complete [ 9744.892706] Lustre: server umount lustre-OST0001 complete [ 9763.461033] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 9773.956855] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9774.414837] LustreError: 340484:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9774.555152] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9779.690777] LustreError: 340485:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9779.711289] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9788.795539] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9789.464813] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9795.082748] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9798.506927] Lustre: 341626:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 9807.186643] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9807.525687] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9814.706014] LustreError: 341980:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9814.739277] LustreError: 341980:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9816.081943] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9819.893715] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 9825.579502] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9830.917205] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 9831.434958] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9839.079309] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9843.022620] Lustre: 343497:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 9850.849564] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9850.857511] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9850.867092] Lustre: Skipped 1 previous similar message [ 9850.871863] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9850.877186] Lustre: Skipped 2 previous similar messages [ 9853.514983] Lustre: server umount lustre-MDT0000 complete [ 9856.482849] LustreError: 340485:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9856.511346] LustreError: 340485:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 9857.149170] LustreError: 343514:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788279050 with bad export cookie 1100603521274639506 [ 9857.152952] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9857.172085] LustreError: 343514:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 9857.520687] Lustre: server umount lustre-MDT0001 complete [ 9872.193200] Lustre: server umount lustre-OST0000 complete [ 9886.735982] Lustre: server umount lustre-OST0001 complete [ 9890.615306] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 9892.843723] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 9894.904377] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 12:11:27 (1788279087) [ 9916.817726] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 9925.328664] loop0: detected capacity change from 0 to 1099511627776 [ 9925.742678] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 9925.747334] *** VALIDATE xfs *** [ 9925.754679] XFS (loop0): Mounting V5 Filesystem [ 9925.771220] XFS (loop0): Ending clean mount [ 9927.251711] loop1: detected capacity change from 0 to 18691697672192 [10032.033778] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [10042.759382] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [10076.422465] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10076.632652] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10076.655327] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10076.740344] Lustre: lustre-MDT0000: new disk, initializing [10076.833816] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10076.840425] Lustre: Skipped 1 previous similar message [10076.859891] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10080.764440] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10090.981256] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10091.098663] Lustre: 348631:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10091.239921] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10091.244522] Lustre: Skipped 1 previous similar message [10091.347080] Lustre: lustre-MDT0001: new disk, initializing [10091.566991] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10091.585837] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10096.328864] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10106.487423] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10113.083102] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10120.518846] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10120.789389] Lustre: lustre-OST0000: new disk, initializing [10120.792743] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10120.799324] Lustre: 350597:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10122.647162] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10122.664735] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10122.782973] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10127.705815] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10137.585356] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10142.350291] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10142.569396] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10147.389704] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10147.685270] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10153.448100] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10153.448594] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10153.451366] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10153.451374] Lustre: Skipped 4 previous similar messages [10153.461496] LustreError: Skipped 1 previous similar message [10153.478123] Lustre: Skipped 4 previous similar messages [10158.566165] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10158.575906] Lustre: Skipped 2 previous similar messages [10164.191169] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10164.403417] Lustre: server umount lustre-OST0000 complete [10168.805204] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10168.823957] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10168.827150] Lustre: Skipped 2 previous similar messages [10173.409590] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10183.135131] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10183.364321] Lustre: server umount lustre-MDT0000 complete [10184.162698] LustreError: 348643:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10184.189870] LustreError: 348643:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10187.422991] LustreError: 348623:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788279381 with bad export cookie 1100603521274641060 [10187.428480] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10187.438321] LustreError: 348623:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10187.717618] Lustre: server umount lustre-MDT0001 complete [10198.611954] XFS (loop0): Unmounting Filesystem [10204.001927] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 12:16:36 (1788279396) [10210.812126] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10211.505389] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10211.516833] Lustre: Skipped 2 previous similar messages [10216.170728] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10224.298458] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10228.842721] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10236.754518] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10242.311536] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10249.888215] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10256.148509] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10264.658849] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10279.908495] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10279.911752] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10279.928267] Lustre: Skipped 1 previous similar message [10279.933993] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10279.934031] Lustre: Skipped 4 previous similar messages [10292.191145] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10292.569453] Lustre: server umount lustre-OST0000 complete [10300.389939] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10300.396735] Lustre: Skipped 1 previous similar message [10303.571236] Lustre: server umount lustre-MDT0000 complete [10305.506545] LustreError: 353156:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10305.531892] LustreError: 353156:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [10308.040914] LustreError: 353143:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788279501 with bad export cookie 1100603521274641935 [10308.066858] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10308.499968] Lustre: server umount lustre-MDT0001 complete [10321.799057] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [10324.745473] Key type lgssc unregistered [10325.129683] LNet: 356797:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10325.139368] LNetError: 356797:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10325.164915] LNet: Removed LNI 192.168.203.107@tcp [10326.142168] Key type .llcrypt unregistered [10326.143830] Key type ._llcrypt unregistered [10360.538936] Key type ._llcrypt registered [10360.540721] Key type .llcrypt registered [10360.679113] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [10361.784760] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10361.831630] alg: No test for adler32 (adler32-zlib) [10362.891844] Lustre: Lustre: Build Version: 2.17.57_82_g90dbd01 [10363.147375] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [10364.895751] Key type lgssc registered [10366.085642] Lustre: Echo OBD driver; http://www.lustre.org/ [10374.851152] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10374.918733] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10376.430831] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10380.270322] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10386.930618] LustreError: 358369:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10386.969695] LustreError: 358369:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10389.436571] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10389.927569] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10395.489657] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10404.570916] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10411.198537] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10418.760235] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10419.194656] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10426.925747] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10436.508069] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10446.304853] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [10446.316986] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10446.342652] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10449.892405] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10449.906827] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10451.239541] Lustre: server umount lustre-MDT0001 complete [10455.012506] LustreError: 358369:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10455.026623] LustreError: 358369:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10460.132125] LustreError: 358370:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10460.148686] LustreError: 358370:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10462.200686] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10462.341550] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [10462.349892] Lustre: Found index 1 for lustre-MDT0001, updating log [10462.437588] Lustre: 358358:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10462.879515] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10467.299881] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10468.338139] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [10470.409767] LustreError: 361968:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [10470.425030] LustreError: 361968:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [10470.440839] LustreError: 358361:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [10470.574833] LustreError: 361971:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.203.107@tcp: cfg command failed: rc = -17 [10470.590726] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [10470.590726] [10775.524421] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10775.526183] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [10775.541274] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10775.562691] Lustre: Skipped 3 previous similar messages [10780.641653] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10780.652953] Lustre: Skipped 2 previous similar messages [10785.763241] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10785.773138] Lustre: Skipped 2 previous similar messages [10786.271963] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10786.538422] Lustre: server umount lustre-MDT0001 complete [10790.882660] LustreError: 358370:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10790.892535] LustreError: 358370:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10796.001614] LustreError: 358366:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10796.020209] LustreError: 358366:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10796.392709] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10796.498155] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [10796.501962] Lustre: Found index 1 for lustre-MDT0001, updating log [10796.537767] Lustre: 358359:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10796.793579] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [10800.682846] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10802.165665] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [10802.676984] LustreError: 370605:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [10802.685374] LustreError: 370605:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [10802.695014] LustreError: 358361:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [10802.698338] LustreError: 358361:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [10802.790139] LustreError: 370614:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [10802.803231] LustreError: 370614:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [10802.811859] LustreError: 370614:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.203.107@tcp: cfg command failed: rc = -17 [10802.825548] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.203.107@tcp [10802.825548] [11108.427982] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.7@tcp (stopping) [11108.446241] Lustre: Skipped 1 previous similar message [11109.345877] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11109.369218] Lustre: Skipped 1 previous similar message [11112.611889] Lustre: server umount lustre-MDT0001 complete [11113.508376] LustreError: 358364:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.7@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11113.542203] LustreError: 358364:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11123.861786] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11124.005432] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11124.016295] Lustre: Found index 1 for lustre-MDT0001, updating log [11124.071552] Lustre: 358357:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11124.430327] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [11129.166596] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [11129.849593] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11130.362509] LustreError: 379433:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11130.368388] LustreError: 379433:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11130.381893] LustreError: 358361:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11130.387481] LustreError: 358361:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11130.427478] LustreError: 379436:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [11130.438918] LustreError: 379436:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.203.107@tcp: cfg command failed: rc = -114 [11130.454281] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [11130.454281] [11452.385695] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11452.386938] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11452.388083] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11452.388089] Lustre: Skipped 2 previous similar messages [11452.398544] Lustre: Skipped 2 previous similar messages [11455.717399] Lustre: server umount lustre-MDT0000 complete [11457.508454] LustreError: 358366:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11457.542880] LustreError: 358366:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 8 previous similar messages [11459.348125] LustreError: 358350:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788280653 with bad export cookie 10116543154296534247 [11459.362224] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11459.364225] LustreError: 358350:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11462.632288] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11465.913062] Lustre: server umount lustre-MDT0001 complete [11470.365464] Lustre: server umount lustre-OST0000 complete [11479.929410] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [11489.009991] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [11522.834179] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 12:38:35 (1788280715) [11528.338397] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11528.612415] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11528.637300] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11528.736800] Lustre: lustre-MDT0000: new disk, initializing [11528.821526] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11528.839400] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11533.588778] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [11543.217635] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11543.308825] Lustre: 390745:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11543.362373] LustreError: 390757:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11543.375704] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11543.377153] LustreError: 390757:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11543.398186] Lustre: Skipped 1 previous similar message [11543.523338] Lustre: lustre-MDT0001: new disk, initializing [11543.627933] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11543.661787] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11543.677292] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11547.824114] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [11556.349274] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11562.505286] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11568.754629] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11568.990808] Lustre: lustre-OST0000: new disk, initializing [11568.995653] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11569.004974] Lustre: 392712:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11569.069354] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11574.649502] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [11575.393053] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11575.412699] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11575.520246] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11584.099611] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11593.194812] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [11593.203813] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11593.223779] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11593.226490] Lustre: Skipped 4 previous similar messages [11593.838844] Lustre: server umount lustre-MDT0000 complete [11597.361417] LustreError: 390739:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788280791 with bad export cookie 10116543154296536200 [11597.365823] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11597.381673] LustreError: 390739:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11597.388082] LustreError: 391557:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [11597.388210] LustreError: 391557:0:(client.c:1394:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@000000003366c8f1 x1875147024103040/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 2 fl Rpc:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [11597.825827] Lustre: server umount lustre-MDT0001 complete [11612.077391] Lustre: server umount lustre-OST0000 complete [11644.326625] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [11654.364623] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [11689.409851] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 11487 sec ========= 12:41:21 (1788280881) [11692.436744] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 12:41:23 (1788280883) === [11697.019493] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 12:41:28 (1788280888) === [11739.222467] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [11741.731727] Key type lgssc unregistered [11742.004529] LNet: 399083:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11742.013657] LNetError: 399083:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11743.079859] LNet: Removed LNI 192.168.203.107@tcp [11743.887801] Key type .llcrypt unregistered [11743.891586] Key type ._llcrypt unregistered