[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 551746610 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.003220] x2apic enabled [ 0.004007] Switched APIC routing to physical x2apic. [ 0.005011] kvm-guest: setup PV IPIs [ 0.008674] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009020] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010009] pid_max: default: 32768 minimum: 301 [ 0.012095] LSM: Security Framework initializing [ 0.013039] Yama: becoming mindful. [ 0.015004] SELinux: Initializing. [ 0.016062] *** VALIDATE selinux *** [ 0.024237] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.028644] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.029157] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030097] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031096] *** VALIDATE tmpfs *** [ 0.032425] *** VALIDATE proc *** [ 0.034092] *** VALIDATE cgroup *** [ 0.035007] *** VALIDATE cgroup2 *** [ 0.036252] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.037145] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.038007] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.039025] Spectre V2 : User space: Vulnerable [ 0.040008] Speculative Store Bypass: Vulnerable [ 0.043451] debug: unmapping init [mem 0xffffffffb6e59000-0xffffffffb6e60fff] [ 0.046127] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.047621] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.048015] ... version: 2 [ 0.049009] ... bit width: 48 [ 0.050006] ... generic registers: 4 [ 0.051007] ... value mask: 0000ffffffffffff [ 0.052010] ... max period: 00007fffffffffff [ 0.053011] ... fixed-purpose events: 3 [ 0.054009] ... event mask: 000000070000000f [ 0.056196] rcu: Hierarchical SRCU implementation. [ 0.058379] smp: Bringing up secondary CPUs ... [ 0.059531] x86: Booting SMP configuration: [ 0.060018] .... node #0, CPUs: #1 #2 #3 [ 0.063226] smp: Brought up 1 node, 4 CPUs [ 0.065013] smpboot: Max logical packages: 1 [ 0.066018] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.246208] node 0 deferred pages initialised in 178ms [ 0.249335] devtmpfs: initialized [ 0.250218] x86/mm: Memory block size: 128MB [ 0.252727] gcov: version magic: 0x41383552 [ 0.254181] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.257124] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.260464] pinctrl core: initialized pinctrl subsystem [ 0.262167] [ 0.262546] ************************************************************* [ 0.264014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.266014] ** ** [ 0.268020] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.270017] ** ** [ 0.274026] ** This means that this kernel is built to expose internal ** [ 0.277013] ** IOMMU data structures, which may compromise security on ** [ 0.280015] ** your system. ** [ 0.282012] ** ** [ 0.285023] ** If you see this message and you are not debugging the ** [ 0.287012] ** kernel, report this immediately to your vendor! ** [ 0.289013] ** ** [ 0.292033] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.294012] ************************************************************* [ 0.296752] NET: Registered protocol family 16 [ 0.298454] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.301064] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.303065] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.307162] cpuidle: using governor menu [ 0.308536] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.309700] PCI: Using configuration type 1 for base access [ 0.313163] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.325061] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.327016] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.330098] cryptd: max_cpu_qlen set to 1000 [ 0.333223] ACPI: Added _OSI(Module Device) [ 0.334012] ACPI: Added _OSI(Processor Device) [ 0.335011] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.337011] ACPI: Added _OSI(Processor Aggregator Device) [ 0.340964] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.348609] ACPI: Interpreter enabled [ 0.349071] ACPI: PM: (supports S0 S3 S4 S5) [ 0.350020] ACPI: Using IOAPIC for interrupt routing [ 0.352116] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.354385] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.365502] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.368048] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.370020] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.373125] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.377486] acpiphp: Slot [2] registered [ 0.378000] acpiphp: Slot [5] registered [ 0.380115] acpiphp: Slot [6] registered [ 0.382101] acpiphp: Slot [7] registered [ 0.383134] acpiphp: Slot [8] registered [ 0.384085] acpiphp: Slot [9] registered [ 0.385108] acpiphp: Slot [10] registered [ 0.386163] acpiphp: Slot [3] registered [ 0.387081] acpiphp: Slot [4] registered [ 0.389092] acpiphp: Slot [11] registered [ 0.390091] acpiphp: Slot [12] registered [ 0.391117] acpiphp: Slot [13] registered [ 0.392104] acpiphp: Slot [14] registered [ 0.394085] acpiphp: Slot [15] registered [ 0.395087] acpiphp: Slot [16] registered [ 0.396113] acpiphp: Slot [17] registered [ 0.397081] acpiphp: Slot [18] registered [ 0.398115] acpiphp: Slot [19] registered [ 0.400083] acpiphp: Slot [20] registered [ 0.401078] acpiphp: Slot [21] registered [ 0.402079] acpiphp: Slot [22] registered [ 0.403104] acpiphp: Slot [23] registered [ 0.405101] acpiphp: Slot [24] registered [ 0.406109] acpiphp: Slot [25] registered [ 0.407083] acpiphp: Slot [26] registered [ 0.408107] acpiphp: Slot [27] registered [ 0.410095] acpiphp: Slot [28] registered [ 0.411117] acpiphp: Slot [29] registered [ 0.412085] acpiphp: Slot [30] registered [ 0.413108] acpiphp: Slot [31] registered [ 0.414108] PCI host bridge to bus 0000:00 [ 0.416022] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.418038] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.419025] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.421025] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.423024] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.425026] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.427277] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.430467] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.433160] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.442021] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.448009] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.450022] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.453026] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.456023] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.459077] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.460820] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.463095] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.465816] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.470017] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.483015] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.488018] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.494073] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.511108] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.519022] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.540022] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.551304] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.589031] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.618021] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.698025] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.713989] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.739023] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.764121] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.808020] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.824766] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.847022] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.855042] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.890025] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.917889] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.942024] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.967023] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 1.029024] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 1.059159] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 1.099034] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 1.124024] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 1.150021] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 1.166108] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 1.168383] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 1.170340] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 1.172330] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 1.174206] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 1.179059] iommu: Default domain type: Passthrough [ 1.181430] SCSI subsystem initialized [ 1.183153] ACPI: bus type USB registered [ 1.184156] usbcore: registered new interface driver usbfs [ 1.186073] usbcore: registered new interface driver hub [ 1.187076] usbcore: registered new device driver usb [ 1.189192] pps_core: LinuxPPS API ver. 1 registered [ 1.190009] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 1.193068] PTP clock support registered [ 1.194314] EDAC MC: Ver: 3.0.0 [ 1.196200] PCI: Using ACPI for IRQ routing [ 1.197913] NetLabel: Initializing [ 1.199009] NetLabel: domain hash size = 128 [ 1.200009] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 1.202141] NetLabel: unlabeled traffic allowed by default [ 1.204206] vgaarb: loaded [ 1.205305] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 1.207028] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 1.216129] clocksource: Switched to clocksource kvm-clock [ 1.332210] VFS: Disk quotas dquot_6.6.0 [ 1.333581] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.336363] *** VALIDATE ramfs *** [ 1.337356] *** VALIDATE hugetlbfs *** [ 1.338707] pnp: PnP ACPI init [ 1.340772] pnp: PnP ACPI: found 6 devices [ 1.356624] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.359448] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.361218] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.363108] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.365029] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.366923] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.369450] NET: Registered protocol family 2 [ 1.371559] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.376210] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.379732] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.383967] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.386890] TCP: Hash tables configured (established 65536 bind 65536) [ 1.389695] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.392936] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.396274] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.400363] NET: Registered protocol family 1 [ 1.403321] RPC: Registered named UNIX socket transport module. [ 1.405792] RPC: Registered udp transport module. [ 1.423915] RPC: Registered tcp transport module. [ 1.428508] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.431972] NET: Registered protocol family 44 [ 1.434889] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.436424] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.437585] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.438918] PCI: CLS 0 bytes, default 64 [ 1.442816] Unpacking initramfs... [ 2.833667] debug: unmapping init [mem 0xffff9f71fcc54000-0xffff9f71fffbffff] [ 2.837098] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.838863] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.841593] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.387021] Initialise system trusted keyrings [ 3.388802] Key type blacklist registered [ 3.390761] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.397231] zbud: loaded [ 3.399694] *** VALIDATE nfs *** [ 3.401212] *** VALIDATE nfs4 *** [ 3.402901] pstore: using deflate compression [ 3.406806] Platform Keyring initialized [ 3.503962] NET: Registered protocol family 38 [ 3.505088] Key type asymmetric registered [ 3.505852] Asymmetric key parser 'x509' registered [ 3.507368] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.510212] io scheduler mq-deadline registered [ 3.510912] io scheduler kyber registered [ 3.512252] io scheduler bfq registered [ 3.514176] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.517172] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.519626] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.521962] ACPI: Power Button [PWRF] [ 3.528215] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.534197] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.545653] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.551366] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.563753] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.591181] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.620978] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.626857] Non-volatile memory driver v1.3 [ 3.628283] Linux agpgart interface v0.103 [ 3.653335] virtio_blk virtio1: [vda] 145880 512-byte logical blocks (74.7 MB/71.2 MiB) [ 3.656510] vda: detected capacity change from 0 to 74690560 [ 3.668445] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.670608] vdb: detected capacity change from 0 to 1073741824 [ 3.685814] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.688076] vdc: detected capacity change from 0 to 2621440000 [ 3.700984] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.703262] vdd: detected capacity change from 0 to 2621440000 [ 3.726990] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.731272] vde: detected capacity change from 0 to 4294967296 [ 3.747219] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.749895] vdf: detected capacity change from 0 to 4294967296 [ 3.756287] libphy: Fixed MDIO Bus: probed [ 3.761326] usbcore: registered new interface driver usbserial_generic [ 3.763578] usbserial: USB Serial support registered for generic [ 3.765635] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.768957] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.770428] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.773168] mousedev: PS/2 mouse device common for all mice [ 3.776745] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.779280] rtc_cmos 00:05: RTC can wake from S4 [ 3.783277] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.787026] rtc_cmos 00:05: registered as rtc0 [ 3.789862] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.792083] intel_pstate: CPU model not supported [ 3.795132] hid: raw HID events driver (C) Jiri Kosina [ 3.797103] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.797220] usbcore: registered new interface driver usbhid [ 3.800942] usbhid: USB HID core driver [ 3.802261] drop_monitor: Initializing network drop monitor service [ 3.804678] Initializing XFRM netlink socket [ 3.806668] NET: Registered protocol family 10 [ 3.809434] Segment Routing with IPv6 [ 3.810986] NET: Registered protocol family 17 [ 3.813356] mpls_gso: MPLS GSO support [ 3.818253] RAS: Correctable Errors collector initialized. [ 3.820020] AVX version of gcm_enc/dec engaged. [ 3.821481] AES CTR mode by8 optimization enabled [ 3.892947] sched_clock: Marking stable (3892931672, 0)->(4769378162, -876446490) [ 3.895530] registered taskstats version 1 [ 3.897335] Loading compiled-in X.509 certificates [ 3.899020] zswap: loaded using pool lzo/zbud [ 3.921586] Key type big_key registered [ 3.933688] Key type encrypted registered [ 3.934841] ima: No TPM chip found, activating TPM-bypass! [ 3.936350] ima: Allocated hash algorithm: sha1 [ 3.937543] ima: No architecture policies found [ 3.938799] evm: Initialising EVM extended attributes: [ 3.940203] evm: security.selinux [ 3.941058] evm: security.ima [ 3.941833] evm: security.capability [ 3.942843] evm: HMAC attrs: 0x1 [ 3.945181] rtc_cmos 00:05: setting system clock to 2026-07-29 07:19:14 UTC (1785309554) [ 3.951214] debug: unmapping init [mem 0xffffffffb7e03000-0xffffffffb7ffffff] [ 3.953633] debug: unmapping init [mem 0xffffffffb6b82000-0xffffffffb6e58fff] [ 3.962258] Write protecting the kernel read-only data: 28672k [ 3.965283] debug: unmapping init [mem 0xffffffffb5203000-0xffffffffb53fffff] [ 3.967558] debug: unmapping init [mem 0xffffffffb5b14000-0xffffffffb5bfffff] [ 3.998992] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 4.006327] systemd[1]: Detected virtualization kvm. [ 4.007975] systemd[1]: Detected architecture x86-64. [ 4.009388] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 4.036295] systemd[1]: No hostname configured. [ 4.038176] systemd[1]: Set hostname to . [ 4.041056] random: systemd: uninitialized urandom read (16 bytes read) [ 4.044023] systemd[1]: Initializing machine ID from random generator. [ 4.082690] random: ln: uninitialized urandom read (6 bytes read) [ 4.177074] random: systemd: uninitialized urandom read (16 bytes read) [ 4.179152] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 4.182943] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ 4.185880] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Swap. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Started Memstrack Anylazing Service. Starting Setup Virtual Console... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. Starting Create Volatile Files and Directories... Starting Journal Service... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Initrd Root Device. Starting Apply Kernel Variables... [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.835887] device-mapper: uevent: version 1.0.3 [ 4.838326] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook...[ 5.597444] random: fast init done [ 5.611546] virtio_net virtio0 ens2: renamed from eth0 [ 5.781212] scsi host0: ata_piix [ 5.805507] scsi host1: ata_piix [ 5.807588] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.810743] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 10.177818] dracut-initqueue[583]: RTNETLINK answers: File exists [ 10.448125] random: crng init done [ 10.449457] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 11.008958] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped target Timers. [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped target Swap. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. Stopping udev Kernel Device Manager... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 12.262502] printk: systemd: 26 output lines suppressed due to ratelimiting [ 12.532596] SELinux: Disabled at runtime. [ 12.597298] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.604580] systemd[1]: Detected virtualization kvm. [ 12.606067] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 13.180990] systemd[1]: initrd-switch-root.service: Succeeded. [ 13.186393] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 13.193144] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 13.196105] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 13.199264] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 13.205956] systemd[1]: Starting Journal Service... Starting Journal Service... [ 13.214786] systemd[1]: Created slice system-sshd\x2dkeygen.slice. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on udev Control Socket. [ OK ] Created slice system-getty.slice. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. Activating swap /dev/disk/by-label/SWAP... Starting Remount Root and Kernel File Systems... [ OK ] Reached target Paths. [ 13.295364] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on Process Core Dump Socket. [ OK ] Reached target rpc_pipefs.target. Starting Apply Kernel Variables... Mounting Huge Pages File System... Mounting POSIX Message Queue File System... Mounting Kernel Debug File System... [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [ OK ] Stopped target Initrd Root File System. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Started Journal Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Activated swap /dev/disk/by-label/SWAP. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Kernel Debug File System. Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 13.680190] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 14.149163] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 14.154614] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 14.252781] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 14.262797] EDAC sbridge: Ver: 1.1.2 [ 15.876268] Key type dns_resolver registered [ 16.198238] NFS: Registering the id_resolver key type [ 16.199905] Key type id_resolver registered [ 16.201437] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. Starting Login Service... Starting Restore /run/initramfs on shutdown... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting Hostname Service... [ OK ] Started Login Service. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting System Logging Service... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg114-server login: [ 62.982724] libcfs: loading out-of-tree module taints kernel. [ 63.070936] Key type ._llcrypt registered [ 63.072399] Key type .llcrypt registered [ 63.287715] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_hostid [ 91.003687] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 93.391044] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 93.418416] alg: No test for adler32 (adler32-zlib) [ 95.185608] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [ 96.416460] LNet: Added LNI 192.168.201.114@tcp [8/256/0/180] [ 98.169503] Key type lgssc registered [ 101.340442] Lustre: Echo OBD driver; http://www.lustre.org/ [ 121.170455] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 147.968063] hrtimer: interrupt took 15065635 ns [ 168.468687] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 184.792569] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 184.842390] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 186.445748] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 186.595944] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 186.856907] Lustre: lustre-MDT0000: new disk, initializing [ 186.957570] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 186.992171] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 193.032533] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 212.769607] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 212.865437] Lustre: 6510:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 212.901664] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 212.907511] Lustre: Skipped 1 previous similar message [ 213.033934] Lustre: lustre-MDT0001: new disk, initializing [ 213.097716] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 213.145803] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 213.164902] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 218.844545] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 224.997411] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 235.925173] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 236.211283] Lustre: lustre-OST0000: new disk, initializing [ 236.220493] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 236.230358] Lustre: 8446:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 236.321327] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 244.392511] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 245.788640] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 245.811933] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 245.914689] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 264.719439] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 265.022369] Lustre: lustre-OST0001: new disk, initializing [ 265.028644] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 265.048642] Lustre: 9519:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 265.206090] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 272.994804] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 273.003746] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 273.163754] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 275.077938] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 287.716641] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 298.774761] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 304.750810] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing check_logdir /tmp/testlogs/ [ 310.765794] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing yml_node [ 315.127313] Lustre: DEBUG MARKER: Client: 2.17.55.22 [ 317.838887] Lustre: DEBUG MARKER: MDS: 2.17.55.22 [ 320.707329] Lustre: DEBUG MARKER: OSS: 2.17.55.22 [ 322.738598] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Wed Jul 29 03:24:30 EDT 2026 [ 341.715409] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 343.330631] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 361.953156] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 361.968891] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 361.998612] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 365.543305] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 365.547756] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 365.564332] Lustre: Skipped 2 previous similar messages [ 365.581162] Lustre: Skipped 3 previous similar messages [ 368.095210] Lustre: server umount lustre-MDT0000 complete [ 372.653486] LustreError: 6502:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785309923 with bad export cookie 12721572353982331765 [ 372.668523] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 375.776993] LustreError: 6516:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 375.782124] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 375.783247] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 375.805435] LustreError: 6516:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 375.860841] Lustre: Skipped 1 previous similar message [ 377.827412] LustreError: 6515:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 377.838242] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 377.858061] LustreError: 6515:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 377.891356] Lustre: Skipped 2 previous similar messages [ 379.384407] Lustre: server umount lustre-MDT0001 complete [ 383.877893] Lustre: server umount lustre-OST0000 complete [ 388.122350] Lustre: server umount lustre-OST0001 complete [ 396.275926] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_hostid [ 407.357933] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 455.893954] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 456.219645] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 456.267214] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 456.412613] Lustre: lustre-MDT0000: new disk, initializing [ 456.549168] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 456.570567] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 461.465957] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 474.938937] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 475.148202] Lustre: 15076:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 475.256451] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 475.263238] Lustre: Skipped 1 previous similar message [ 475.441187] Lustre: lustre-MDT0001: new disk, initializing [ 475.636311] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 475.733386] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 475.744955] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 480.899232] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 493.511373] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 500.228861] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 509.587154] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 509.863670] Lustre: lustre-OST0000: new disk, initializing [ 509.870074] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 509.878800] Lustre: 17047:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 510.028116] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 511.401624] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 511.415332] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 511.629992] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 519.554218] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 531.184403] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 536.564586] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 536.760919] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 543.717585] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 544.065505] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 547.305950] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 547.310745] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 547.316156] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 547.337409] Lustre: Skipped 1 previous similar message [ 552.741412] Lustre: server umount lustre-OST0000 complete [ 557.539689] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 557.540415] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 557.584930] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 557.584940] Lustre: Skipped 3 previous similar messages [ 564.011978] Lustre: server umount lustre-MDT0000 complete [ 567.796401] LustreError: 15086:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 567.807942] LustreError: 15086:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 569.193382] LustreError: 15071:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785310119 with bad export cookie 12721572353982333298 [ 569.201835] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 569.203054] LustreError: 15071:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 569.468109] Lustre: server umount lustre-MDT0001 complete [ 577.415274] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 03:28:44 (1785310124) [ 584.534897] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 585.170635] LustreError: 19316:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 585.332091] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 590.308708] LustreError: 19317:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 591.474325] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 600.551861] LustreError: 19316:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 600.571883] LustreError: 19316:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 600.790126] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 605.705621] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 614.132341] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 621.357758] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 631.852594] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 642.302615] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 652.050169] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 662.410316] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 675.845261] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 675.858379] LustreError: 19316:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 675.890238] Lustre: Skipped 3 previous similar messages [ 675.970909] LustreError: 19316:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 676.096812] Lustre: server umount lustre-MDT0000 complete [ 682.965740] LustreError: 19296:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785310233 with bad export cookie 12721572353982334180 [ 682.981247] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 683.006428] LustreError: 19296:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 684.342965] Lustre: server umount lustre-MDT0001 complete [ 699.840598] Lustre: server umount lustre-OST0000 complete [ 734.227941] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 753.378797] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 754.240721] LustreError: 23727:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 754.277064] LustreError: 23727:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 754.338688] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 754.365465] Lustre: Skipped 2 previous similar messages [ 761.500746] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 773.570305] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 780.104372] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 783.378653] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 790.782387] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 799.002643] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 810.583871] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 810.872803] Lustre: lustre-OST0001: new disk, initializing [ 810.878658] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 810.903232] Lustre: 26046:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 812.477745] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 812.488227] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 812.708778] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 818.232179] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 829.876878] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 834.882225] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 853.483547] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 853.493309] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 853.513563] Lustre: Skipped 3 previous similar messages [ 853.555133] Lustre: Skipped 6 previous similar messages [ 858.253469] Lustre: server umount lustre-MDT0000 complete [ 858.611532] LustreError: 23727:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 858.651381] LustreError: 23727:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 862.403316] LustreError: 26045:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785310412 with bad export cookie 12721572353982335027 [ 862.406285] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 862.426891] LustreError: 26045:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 863.489302] Lustre: server umount lustre-MDT0001 complete [ 878.988507] Lustre: server umount lustre-OST0000 complete [ 880.096475] Lustre: 3643:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785310414/real 1785310414] req@ffff9f727589a300 x1872032847631872/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785310430 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 883.779471] Lustre: server umount lustre-OST0001 complete [ 901.922454] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 913.972153] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 914.487991] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 914.494525] Lustre: Skipped 3 previous similar messages [ 921.210940] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 931.433639] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 939.622863] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 943.517431] Lustre: 30164:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 951.116601] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 959.662447] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 970.885893] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 978.731467] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 988.917600] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 993.402563] Lustre: 32045:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1001.962408] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1001.963745] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1001.985517] Lustre: Skipped 3 previous similar messages [ 1002.003739] Lustre: Skipped 2 previous similar messages [ 1006.824991] Lustre: server umount lustre-MDT0000 complete [ 1007.082102] LustreError: 29013:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1007.111413] LustreError: 29013:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 14 previous similar messages [ 1010.943516] LustreError: 29000:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785310561 with bad export cookie 12721572353982336322 [ 1010.967086] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1010.967650] LustreError: 29000:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1011.400414] Lustre: server umount lustre-MDT0001 complete [ 1025.408064] Lustre: server umount lustre-OST0000 complete [ 1028.384789] Lustre: 3641:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785310562/real 1785310562] req@ffff9f727382f800 x1872032847688064/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785310578 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1029.437788] Lustre: server umount lustre-OST0001 complete [ 1040.448846] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 03:36:28 (1785310588) [ 1073.893803] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 1086.164255] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1091.908225] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 1102.106885] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1108.075613] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 1111.406989] Lustre: 35999:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1119.196554] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1127.429278] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 1137.739409] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1146.165845] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 1155.343825] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1159.273179] Lustre: 37876:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1168.867584] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1168.874333] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1168.901032] Lustre: Skipped 4 previous similar messages [ 1168.910764] Lustre: Skipped 2 previous similar messages [ 1172.157567] Lustre: server umount lustre-MDT0000 complete [ 1176.577438] LustreError: 34831:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785310727 with bad export cookie 12721572353982337428 [ 1176.580229] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1176.588581] LustreError: 34831:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1177.202382] Lustre: server umount lustre-MDT0001 complete [ 1192.512378] Lustre: server umount lustre-OST0000 complete [ 1194.464092] Lustre: 3642:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785310729/real 1785310729] req@ffff9f7143bf3800 x1872032847740288/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785310745 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1196.486200] Lustre: server umount lustre-OST0001 complete [ 1205.638916] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 03:39:13 (1785310753) [ 1226.535060] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 1240.894572] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1241.200495] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1241.208225] Lustre: Skipped 7 previous similar messages [ 1247.048922] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 1256.742033] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1263.494490] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 1267.445778] Lustre: 41169:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1274.726213] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1281.530247] LustreError: 41522:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1281.563774] LustreError: 41522:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 17 previous similar messages [ 1284.465155] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 1299.790364] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1308.786606] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 1316.657478] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1321.114922] Lustre: 43047:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1322.282771] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1322.295450] Lustre: Skipped 1 previous similar message [ 1326.726244] Lustre: lustre-MDT0000: Not available for connect from 192.168.201.14@tcp (stopping) [ 1328.610945] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1328.620698] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1328.638322] Lustre: Skipped 2 previous similar messages [ 1332.076769] Lustre: server umount lustre-MDT0000 complete [ 1336.433778] LustreError: 43049:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785310886 with bad export cookie 12721572353982338534 [ 1336.449064] LustreError: 43049:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1336.450098] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1343.279637] Lustre: server umount lustre-MDT0001 complete [ 1352.991918] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1362.464943] LustreError: 3640:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff9f724f517800 x1872032847801728/t0(0) o250->MGC192.168.201.114@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1368.651596] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 1372.358401] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1381.146892] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1386.990412] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 1386.991950] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1387.010787] Lustre: Skipped 1 previous similar message [ 1397.562267] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1400.295566] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1407.896478] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1410.102368] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1422.306046] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1424.439503] Lustre: server umount lustre-MDT0000 complete [ 1429.078634] LustreError: 39999:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785310979 with bad export cookie 12721572353982339675 [ 1429.086697] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1429.102334] LustreError: 39999:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1429.571804] Lustre: server umount lustre-MDT0001 complete [ 1444.012808] Lustre: server umount lustre-OST0000 complete [ 1459.610242] Lustre: server umount lustre-OST0001 complete [ 1469.083392] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 03:43:37 (1785311017) [ 1488.284768] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 1499.746682] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1505.875928] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 1515.236803] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1520.243054] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 1524.101540] Lustre: 48818:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1532.132289] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1541.474302] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 1551.211834] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1559.614951] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 1568.422759] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1573.805193] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1587.689593] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1587.709774] Lustre: Skipped 11 previous similar messages [ 1587.756264] Lustre: server umount lustre-MDT0000 complete [ 1592.159238] LustreError: 48421:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785311142 with bad export cookie 12721572353982340494 [ 1592.160513] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1592.169439] LustreError: 48421:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1592.688517] Lustre: server umount lustre-MDT0001 complete [ 1607.269104] Lustre: server umount lustre-OST0000 complete [ 1609.195056] Lustre: 3643:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785311143/real 1785311143] req@ffff9f727c2c1f80 x1872032847901056/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785311159 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1612.096561] Lustre: server umount lustre-OST0001 complete [ 1622.526852] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 03:46:10 (1785311170) [ 1625.288595] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1627.182235] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 03:46:15 (1785311175) [ 1639.598729] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_hostid [ 1650.374640] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 1676.980990] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1677.362355] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1677.431779] Lustre: lustre-MDT0000: new disk, initializing [ 1677.559347] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1683.169316] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 1693.602434] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1700.776134] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1701.032853] Lustre: lustre-OST0000: new disk, initializing [ 1701.036383] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1701.039390] Lustre: Skipped 1 previous similar message [ 1701.048962] Lustre: 54896:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1703.052398] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1703.061096] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1703.092345] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1707.636987] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 1719.028336] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1721.581183] Lustre: DEBUG MARKER: create test files [ 1754.596868] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1754.602651] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1754.631400] Lustre: Skipped 16 previous similar messages [ 1759.976700] Lustre: server umount lustre-OST0000 complete [ 1764.210651] Lustre: server umount lustre-MDT0000 complete [ 1775.562876] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing unload_modules_local [ 1778.318421] Key type lgssc unregistered [ 1778.597748] LNet: 56531:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1778.607347] LNetError: 56531:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1779.624357] LNet: Removed LNI 192.168.201.114@tcp [ 1780.528153] Key type .llcrypt unregistered [ 1780.535210] Key type ._llcrypt unregistered [ 1808.604560] Key type ._llcrypt registered [ 1808.607541] Key type .llcrypt registered [ 1808.714673] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 1809.880590] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1810.061436] alg: No test for adler32 (adler32-zlib) [ 1811.096389] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [ 1811.293408] LNet: Added LNI 192.168.201.114@tcp [8/256/0/180] [ 1812.954492] Key type lgssc registered [ 1813.972208] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1820.816499] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1820.839747] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1822.380422] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1827.426162] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 1835.944692] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1843.696212] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1844.405251] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1847.503604] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:98 to 0x240000400:129) [ 1847.709466] Lustre: 59243:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1847.726125] Lustre: 59243:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1847.731645] Lustre: 59243:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1847.740062] Lustre: 59243:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1847.751573] Lustre: 59243:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1847.769045] Lustre: 59243:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1851.817693] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 1861.250514] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1866.556041] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1871.924732] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1883.623824] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1883.635203] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1883.658583] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1888.737065] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1892.832121] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1893.305444] Lustre: server umount lustre-OST0000 complete [ 1900.588934] Lustre: server umount lustre-MDT0000 complete [ 1916.232784] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing unload_modules_local [ 1919.439350] Key type lgssc unregistered [ 1919.912713] LNet: 60858:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1919.917580] LNetError: 60858:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1920.934774] LNet: Removed LNI 192.168.201.114@tcp [ 1922.159055] Key type .llcrypt unregistered [ 1922.162536] Key type ._llcrypt unregistered [ 1959.298752] Key type ._llcrypt registered [ 1959.301819] Key type .llcrypt registered [ 1959.488415] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 1961.165938] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1961.220889] alg: No test for adler32 (adler32-zlib) [ 1962.299761] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [ 1962.593936] LNet: Added LNI 192.168.201.114@tcp [8/256/0/180] [ 1964.336157] Key type lgssc registered [ 1965.440571] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1973.577736] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1973.590791] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1975.198308] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1980.130617] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 1988.176952] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1995.147568] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1995.616310] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1999.741038] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1999.841765] Lustre: 63693:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1999.851535] Lustre: 63693:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1999.859570] Lustre: 63693:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1999.868828] Lustre: 63693:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1999.876872] Lustre: 63693:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1999.888682] Lustre: 63693:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 2002.294450] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 2009.464775] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2016.277690] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 2025.455806] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2025.463236] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2025.486931] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2030.625453] Lustre: server umount lustre-OST0000 complete [ 2034.865217] Lustre: server umount lustre-MDT0000 complete [ 2046.092516] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing unload_modules_local [ 2049.076501] Key type lgssc unregistered [ 2049.395817] LNet: 65252:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2049.401973] LNetError: 65252:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2049.415559] LNet: Removed LNI 192.168.201.114@tcp [ 2050.442737] Key type .llcrypt unregistered [ 2050.444897] Key type ._llcrypt unregistered [ 2071.422800] Key type ._llcrypt registered [ 2071.427591] Key type .llcrypt registered [ 2071.538400] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_hostid [ 2088.964582] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 2090.607529] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2090.715760] alg: No test for adler32 (adler32-zlib) [ 2091.817542] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [ 2092.094109] LNet: Added LNI 192.168.201.114@tcp [8/256/0/180] [ 2093.728200] Key type lgssc registered [ 2095.227383] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2132.242191] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2132.280853] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2133.431720] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2133.448657] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2133.519705] Lustre: lustre-MDT0000: new disk, initializing [ 2133.590467] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2133.629064] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2138.909179] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 2152.573347] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2152.704305] Lustre: 68404:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2152.757727] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2152.760793] Lustre: Skipped 1 previous similar message [ 2152.818853] Lustre: lustre-MDT0001: new disk, initializing [ 2152.910282] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2152.950386] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2152.971579] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2158.811981] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 2169.245820] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2175.465187] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2181.833113] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2182.084968] Lustre: lustre-OST0000: new disk, initializing [ 2182.088313] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2182.094596] Lustre: 70374:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2182.220700] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2189.313765] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 2191.945935] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2191.967154] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2192.060638] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2199.259097] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2203.982329] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2204.216513] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2209.403191] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2209.597303] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2212.843197] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2212.849687] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2212.853189] LustreError: Skipped 1 previous similar message [ 2212.855160] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2212.896779] Lustre: Skipped 1 previous similar message [ 2217.946950] Lustre: server umount lustre-OST0000 complete [ 2223.074987] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2223.075683] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2223.088067] Lustre: Skipped 1 previous similar message [ 2223.095029] Lustre: Skipped 1 previous similar message [ 2227.924768] Lustre: server umount lustre-MDT0000 complete [ 2231.854384] LustreError: 68397:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785311782 with bad export cookie 1271841923248510757 [ 2231.858996] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2231.866943] LustreError: 68397:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2232.171558] Lustre: server umount lustre-MDT0001 complete [ 2239.957858] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 03:56:28 (1785311788) [ 2250.114775] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 2287.845038] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_hostid [ 2296.057769] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 2329.842414] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2330.028924] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2330.047940] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2330.109677] Lustre: lustre-MDT0000: new disk, initializing [ 2330.172822] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2330.189276] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2334.516864] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 2345.010316] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2345.198212] Lustre: 76392:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2345.311700] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2345.321251] Lustre: Skipped 1 previous similar message [ 2345.469488] Lustre: lustre-MDT0001: new disk, initializing [ 2345.647743] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2345.706462] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2345.724365] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2349.848977] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 2358.554574] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2364.158745] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2371.649960] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2372.113159] Lustre: lustre-OST0000: new disk, initializing [ 2372.120809] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2372.131388] Lustre: 78363:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2372.300169] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2378.986810] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 2379.313223] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2379.321408] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2379.387205] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2389.228313] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2394.042433] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2394.217092] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2398.068513] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2398.209918] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2399.716850] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2399.727853] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2399.733741] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2399.733748] Lustre: Skipped 1 previous similar message [ 2399.766925] Lustre: Skipped 1 previous similar message [ 2404.837304] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2404.850303] Lustre: Skipped 2 previous similar messages [ 2405.787638] Lustre: server umount lustre-OST0000 complete [ 2409.959108] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2409.977223] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2409.978899] Lustre: Skipped 1 previous similar message [ 2420.196922] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2420.205400] Lustre: Skipped 4 previous similar messages [ 2423.264098] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2423.453595] Lustre: server umount lustre-MDT0000 complete [ 2425.313910] LustreError: 77206:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2425.335447] LustreError: 77206:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 2427.435892] LustreError: 77200:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785311977 with bad export cookie 1271841923248511639 [ 2427.441698] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2427.453603] LustreError: 77200:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2427.953310] Lustre: server umount lustre-MDT0001 complete [ 2436.372094] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 03:59:44 (1785311984) [ 2442.781759] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2443.136299] LustreError: 80776:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2443.193026] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2447.548772] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 2448.360757] LustreError: 80777:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2452.464218] LustreError: 80776:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2455.875780] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2460.085910] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 2468.656237] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2474.591157] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2481.421922] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2481.941769] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2481.944154] Lustre: Skipped 1 previous similar message [ 2488.181533] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 2495.496379] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2499.018855] Lustre: 83445:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting d37905e9-c921-4609-83e8-c04318cd4b11 at adminstrative request [ 2501.414733] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2501.434106] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2510.835618] Lustre: MGS: Client d37905e9-c921-4609-83e8-c04318cd4b11 (at 0@lo) reconnecting [ 2510.853691] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x11a67d7d9dad5e09 to 0x11a67d7d9dad60a9 [ 2510.868146] Lustre: MGC192.168.201.114@tcp: Connection restored to 0@lo (at 0@lo) [ 2538.018536] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2538.255818] Lustre: lustre-OST0001: new disk, initializing [ 2538.261996] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2538.269608] Lustre: 83851:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2546.280270] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 2546.779409] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2546.790780] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2546.918609] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2556.644110] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2562.529471] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2562.534865] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2562.540641] LustreError: Skipped 1 previous similar message [ 2562.544315] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2562.556053] Lustre: Skipped 1 previous similar message [ 2562.577383] Lustre: Skipped 1 previous similar message [ 2565.590769] Lustre: server umount lustre-OST0001 complete [ 2567.664735] LustreError: 82595:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2567.690370] LustreError: 82595:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2570.209882] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2572.779260] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2572.793327] Lustre: Skipped 1 previous similar message [ 2575.644972] Lustre: server umount lustre-OST0000 complete [ 2586.215364] Lustre: server umount lustre-MDT0000 complete [ 2588.131425] LustreError: 81504:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2588.157907] LustreError: 81504:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2590.695621] LustreError: 83850:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785312141 with bad export cookie 1271841923248513193 [ 2590.706784] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2590.712104] LustreError: 83850:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2591.421278] Lustre: server umount lustre-MDT0001 complete [ 2602.481641] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 04:02:30 (1785312150) [ 2604.210866] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2606.363227] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 04:02:34 (1785312154) [ 2608.408454] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2610.493656] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 04:02:38 (1785312158) [ 2612.170025] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2614.028198] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 04:02:42 (1785312162) [ 2615.434291] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2616.844965] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 04:02:45 (1785312165) [ 2618.101180] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2623.997385] loop: module loaded [ 2624.021415] loop0: detected capacity change from 0 to 4294967296 [ 2624.070352] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2625.792930] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2630.722899] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2634.528326] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2659.733239] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2659.766384] LustreError: 87977:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2659.770350] LustreError: 87977:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2659.775011] LustreError: 87977:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2659.779867] LustreError: 87977:0:(super25.c:178:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2678.653740] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_hostid [ 2688.009553] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 2723.508905] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2723.724090] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2723.743828] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2723.817124] Lustre: lustre-MDT0000: new disk, initializing [ 2723.857426] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2723.859476] Lustre: Skipped 1 previous similar message [ 2723.882122] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2727.980902] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 2738.265212] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2738.374188] Lustre: 90843:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2738.657791] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2742.746745] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 2752.004226] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2756.967201] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2763.272765] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2763.456418] Lustre: 92811:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2769.536261] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 2771.499976] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2771.652769] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2780.626439] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2786.728296] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2787.011196] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2792.836480] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2793.040674] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2797.027505] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2797.030146] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2797.052716] Lustre: Skipped 2 previous similar messages [ 2797.059784] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2797.061787] Lustre: Skipped 5 previous similar messages [ 2801.848584] Lustre: server umount lustre-OST0000 complete [ 2812.525927] Lustre: server umount lustre-MDT0000 complete [ 2812.897324] LustreError: 90852:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2812.917548] LustreError: 90852:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2816.986564] LustreError: 90835:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785312367 with bad export cookie 1271841923248513809 [ 2816.993642] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2816.999738] LustreError: 90835:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2817.536982] Lustre: server umount lustre-MDT0001 complete [ 2825.286289] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 04:06:13 (1785312373) [ 2831.987531] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2836.799994] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 2844.710615] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2848.964421] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 2855.191546] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2861.163184] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2867.621929] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2867.884414] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2867.889949] Lustre: Skipped 4 previous similar messages [ 2874.673527] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 2883.309445] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2888.118271] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2888.286307] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2893.104617] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2893.378820] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2899.602410] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2899.835942] Lustre: lustre-OST0001: new disk, initializing [ 2899.838655] Lustre: Skipped 2 previous similar messages [ 2899.840979] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2899.845884] Lustre: Skipped 3 previous similar messages [ 2899.849955] Lustre: 98593:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2905.260296] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 2909.379910] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2909.400972] Lustre: Skipped 2 previous similar messages [ 2909.419649] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2909.514700] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2915.640169] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2919.887780] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2920.065121] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2924.463253] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2924.686040] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2943.075475] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2961.413587] Lustre: Failing over lustre-MDT0000 [ 2961.787327] Lustre: server umount lustre-MDT0000 complete [ 2962.558159] LustreError: 99846:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.201.14@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2962.567681] LustreError: 99846:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 2962.917382] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2962.925357] LustreError: Skipped 1 previous similar message [ 2962.928371] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2962.935547] Lustre: Skipped 3 previous similar messages [ 2982.378561] Lustre: 66627:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785312516/real 1785312516] req@000000007c77c87d x1872034940592384/t0(0) o400->MGC192.168.201.114@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1785312532 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2982.405393] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2984.931465] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2984.933695] LDISKFS-fs (dm-0): recovery complete [ 2984.957484] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2991.593028] LustreError: 66626:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000da5341fa x1872034940600576/t0(0) o250->MGC192.168.201.114@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2992.137065] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2994.259963] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2994.263216] Lustre: 99846:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2996.495577] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 2997.242680] Lustre: 95222:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2997.249576] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2997.258066] Lustre: 95222:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 1 previous similar message [ 2997.343618] LustreError: 101443:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2997.683159] LustreError: 101443:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2998.014512] LustreError: 101443:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2998.033612] LustreError: 101443:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 2998.361976] LustreError: 101443:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2998.367525] LustreError: 101443:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 2998.377128] Lustre: 101443:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2998.388157] Lustre: 101443:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 7 previous similar messages [ 2999.024964] LustreError: 101443:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2999.039882] LustreError: 101443:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 2 previous similar messages [ 2999.370766] LustreError: 101443:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2999.374754] LustreError: 101443:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 2 previous similar messages [ 3000.649768] Lustre: 101443:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3000.664650] Lustre: 101443:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 6 previous similar messages [ 3001.320176] LustreError: 101443:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3001.326582] LustreError: 101443:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 6 previous similar messages [ 3001.656128] LustreError: 101443:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3001.664592] LustreError: 101443:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 6 previous similar messages [ 3004.801123] Lustre: 101443:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3004.809961] Lustre: 101443:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 12 previous similar messages [ 3005.452079] LustreError: 101443:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3005.462439] LustreError: 101443:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 3005.784111] LustreError: 101443:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3005.787038] LustreError: 101443:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 3013.097083] Lustre: 101443:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3013.105530] Lustre: 101443:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 25 previous similar messages [ 3013.754976] LustreError: 101443:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3013.773562] LustreError: 101443:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 25 previous similar messages [ 3014.096128] LustreError: 101443:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3014.103649] LustreError: 101443:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 25 previous similar messages [ 3029.259956] Lustre: 101443:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3029.276381] Lustre: 101443:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 50 previous similar messages [ 3029.915968] LustreError: 101443:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3029.931540] LustreError: 101443:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 50 previous similar messages [ 3030.248125] LustreError: 101443:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3030.251087] LustreError: 101443:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 50 previous similar messages [ 3054.225156] Lustre: 101443:0:(ldlm_lib.c:2393:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3054.234572] Lustre: 101443:0:(ldlm_lib.c:2403:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3054.243463] Lustre: 101443:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 298a412a-fae6-4e59-a7f9-1767cd74b9d2@192.168.201.14@tcp [ 3054.266568] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3054.282356] Lustre: 101443:0:(ldlm_lib.c:1913:abort_req_replay_queue()) @@@ aborted: req@00000000b60b92d5 x1872034921517440/t0(8589934776) o35->298a412a-fae6-4e59-a7f9-1767cd74b9d2@192.168.201.14@tcp:355/0 lens 392/0 e 0 to 0 dl 1785312615 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3054.317602] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3054.338908] Lustre: lustre-MDT0000: Denying connection for new client 298a412a-fae6-4e59-a7f9-1767cd74b9d2 (at 192.168.201.14@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3054.355202] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3054.415490] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3054.417723] Lustre: Skipped 2 previous similar messages [ 3054.437732] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3054.501879] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 3054.521987] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 3068.901572] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3068.922477] Lustre: Skipped 3 previous similar messages [ 3068.930395] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3068.936177] Lustre: Skipped 3 previous similar messages [ 3081.184315] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3081.483875] Lustre: server umount lustre-OST0000 complete [ 3084.275931] LustreError: 97044:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3084.300471] LustreError: 97044:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 42 previous similar messages [ 3085.793421] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3091.130240] Lustre: server umount lustre-OST0001 complete [ 3100.895871] Lustre: server umount lustre-MDT0000 complete [ 3104.034804] LustreError: 95203:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785312654 with bad export cookie 1271841923248565028 [ 3104.042072] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3104.372488] Lustre: server umount lustre-MDT0001 complete [ 3111.587463] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 04:11:00 (1785312660) [ 3118.286377] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3118.295832] Lustre: *** cfs_fail_loc=197, val=0*** [ 3133.856589] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3138.976445] LustreError: 103419:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.114@tcp: failed processing log, type 4: rc = -110 [ 3144.117219] Lustre: *** cfs_fail_loc=197, val=0*** [ 3144.123103] Lustre: Skipped 65 previous similar messages [ 3164.640335] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3164.645695] Lustre: Skipped 2 previous similar messages [ 3170.069759] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 3176.735282] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3178.552367] Lustre: server umount lustre-OST0000 complete [ 3185.955101] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 04:12:14 (1785312734) [ 3204.511900] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3208.241517] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 3215.333134] LustreError: 105036:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3215.338270] LustreError: 105036:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 3216.566391] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3220.155584] LustreError: 105066:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3220.167664] LustreError: 105066:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3220.176542] LustreError: 105066:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 15, retries 0, failed: rc = -5 [ 3220.453587] Lustre: server umount lustre-MDT0000 complete [ 3247.548808] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_hostid [ 3255.400694] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 3286.733829] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3287.080452] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3287.108756] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3287.191411] Lustre: lustre-MDT0000: new disk, initializing [ 3287.332574] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3291.322508] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 3300.097954] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3300.208598] Lustre: 109238:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3300.397277] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3304.668148] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 3312.153948] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3316.536492] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3321.357822] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3321.675581] Lustre: 111207:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3327.244969] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 3330.771676] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3334.818858] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3339.177982] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3339.401444] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3343.593681] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3343.712724] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3346.405518] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3346.415385] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3346.417408] Lustre: Skipped 5 previous similar messages [ 3346.420769] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3346.420775] Lustre: Skipped 12 previous similar messages [ 3346.457291] LustreError: Skipped 2 previous similar messages [ 3351.658737] Lustre: server umount lustre-OST0000 complete [ 3363.903284] Lustre: server umount lustre-MDT0000 complete [ 3369.939457] LustreError: 109274:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785312920 with bad export cookie 1271841923248575682 [ 3369.944368] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3369.963539] LustreError: 109274:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3370.026680] LustreError: 66630:0:(import.c:717:ptlrpc_connect_import_locked()) already connecting [ 3370.619203] Lustre: server umount lustre-MDT0001 complete [ 3385.634857] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 04:15:32 (1785312932) [ 3405.594282] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing unload_modules_local [ 3412.178281] Key type lgssc unregistered [ 3412.776467] LNet: 113715:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3412.781339] LNetError: 113715:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3412.796712] LNet: Removed LNI 192.168.201.114@tcp [ 3413.893587] Key type .llcrypt unregistered [ 3413.895420] Key type ._llcrypt unregistered [ 3463.281124] Key type ._llcrypt registered [ 3463.294373] Key type .llcrypt registered [ 3463.568317] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_hostid [ 3489.558882] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 3491.833492] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3491.866922] alg: No test for adler32 (adler32-zlib) [ 3493.440545] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [ 3494.091603] LNet: Added LNI 192.168.201.114@tcp [8/256/0/180] [ 3495.866123] Key type lgssc registered [ 3498.399689] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3553.677476] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3553.727910] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3555.151721] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3555.197902] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3555.304609] Lustre: lustre-MDT0000: new disk, initializing [ 3555.523835] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3555.562057] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3561.241629] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 3575.040598] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3575.503880] Lustre: lustre-OST0000: new disk, initializing [ 3575.510952] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3575.516571] Lustre: Skipped 1 previous similar message [ 3575.530906] Lustre: 117898:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3575.739152] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3583.160845] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3583.167221] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3583.200218] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3583.949893] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 3597.906146] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3598.124272] Lustre: lustre-OST0001: new disk, initializing [ 3598.130540] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3598.148735] Lustre: 118773:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3598.289427] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3605.775507] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 3607.304205] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3607.312257] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3607.371705] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3617.766846] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3617.775286] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3617.801536] Lustre: Skipped 1 previous similar message [ 3617.828473] Lustre: Skipped 1 previous similar message [ 3622.612896] Lustre: server umount lustre-MDT0000 complete [ 3632.182864] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3639.268576] Lustre: 115099:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785313173/real 1785313173] req@00000000397610a3 x1872036410451456/t0(0) o400->MGC192.168.201.114@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1785313189 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3639.303901] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3640.094697] Lustre: server umount lustre-OST0000 complete [ 3646.889442] Lustre: server umount lustre-OST0001 complete [ 3658.208883] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 04:20:05 (1785313205) [ 3684.694769] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 04:20:32 (1785313232) [ 3696.789693] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 3725.767735] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 04:21:13 (1785313273) [ 3778.662489] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_hostid [ 3789.885806] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 3834.121169] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3834.359937] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3834.379826] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3834.437174] Lustre: lustre-MDT0000: new disk, initializing [ 3834.483022] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3834.494777] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3840.224797] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 3852.942873] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3853.127770] Lustre: 126533:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3853.240809] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3853.247728] Lustre: Skipped 1 previous similar message [ 3853.414572] Lustre: lustre-MDT0001: new disk, initializing [ 3853.610228] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3853.662518] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3853.680773] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3859.353909] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 3870.795648] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3877.514193] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3886.804967] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3887.153202] Lustre: lustre-OST0000: new disk, initializing [ 3887.161352] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3887.170178] Lustre: 128501:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3887.373587] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3895.282217] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 3897.416666] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3897.435579] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3897.577674] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3908.037657] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3917.315463] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3917.637977] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3927.004307] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3927.506077] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3933.668598] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3933.673849] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3933.706948] Lustre: Skipped 1 previous similar message [ 3933.733982] Lustre: Skipped 1 previous similar message [ 3936.861460] Lustre: server umount lustre-OST0000 complete [ 3943.907587] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3943.920713] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3949.027610] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3949.040650] Lustre: Skipped 2 previous similar messages [ 3949.620680] Lustre: server umount lustre-MDT0000 complete [ 3955.020105] LustreError: 126526:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785313505 with bad export cookie 17890516657582173997 [ 3955.056126] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3955.057307] LustreError: 126526:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3956.164176] Lustre: server umount lustre-MDT0001 complete [ 3968.447862] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 04:25:16 (1785313516) [ 3978.563776] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3979.298724] LustreError: 130918:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3979.326938] LustreError: 130918:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3979.416094] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3984.878071] LustreError: 130919:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3985.216477] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 3988.974373] LustreError: 130918:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3994.082640] LustreError: 130919:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3996.032066] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4002.101558] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 4011.744901] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4018.495635] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4027.888480] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4028.120572] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4028.126039] Lustre: Skipped 1 previous similar message [ 4036.849640] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 4050.570289] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4056.327850] Lustre: *** cfs_fail_loc=159, val=0*** [ 4071.545859] Lustre: lustre-MDT0000: Client 018c4320-a05c-493e-a43c-ced1ed11e28c (at 192.168.201.14@tcp) reconnecting [ 4074.308718] Lustre: *** cfs_fail_loc=159, val=0*** [ 4074.321102] Lustre: Skipped 5 previous similar messages [ 4089.539515] Lustre: lustre-MDT0000: Client 018c4320-a05c-493e-a43c-ced1ed11e28c (at 192.168.201.14@tcp) reconnecting [ 4100.066129] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4100.076438] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4100.091337] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4100.131529] Lustre: Skipped 2 previous similar messages [ 4105.808807] Lustre: server umount lustre-OST0000 complete [ 4113.889340] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4113.900084] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4113.913137] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4113.917503] Lustre: Skipped 3 previous similar messages [ 4125.152125] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4125.408035] Lustre: server umount lustre-MDT0000 complete [ 4125.666939] LustreError: 133522:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4125.680489] LustreError: 133522:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4130.544834] LustreError: 130901:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785313681 with bad export cookie 17890516657582174879 [ 4130.545862] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4130.580764] LustreError: 130901:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4131.108860] Lustre: server umount lustre-MDT0001 complete [ 4143.758660] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing unload_modules_local [ 4146.929384] Key type lgssc unregistered [ 4147.381455] LNet: 134701:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4147.393334] LNetError: 134701:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4147.411890] LNet: Removed LNI 192.168.201.114@tcp [ 4148.859120] Key type .llcrypt unregistered [ 4148.863339] Key type ._llcrypt unregistered [ 4183.624505] Key type ._llcrypt registered [ 4183.629878] Key type .llcrypt registered [ 4183.707427] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 4184.797732] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4184.905880] alg: No test for adler32 (adler32-zlib) [ 4187.202872] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [ 4187.863125] LNet: Added LNI 192.168.201.114@tcp [8/256/0/180] [ 4189.649875] Key type lgssc registered [ 4191.545923] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4202.933266] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4202.954715] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4204.458379] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4210.622462] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 4213.739945] LustreError: 136116:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4213.763358] LustreError: 136116:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4218.853364] LustreError: 136117:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4222.279319] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4223.245584] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4232.765990] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 4244.725308] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4255.015697] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4268.478835] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4269.063814] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4275.218203] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4278.329990] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 4288.874898] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4303.686362] Lustre: *** cfs_fail_loc=159, val=0*** [ 4319.918584] Lustre: lustre-MDT0000: Client 65870942-8f79-4461-af4e-0dbeb4f604ef (at 192.168.201.14@tcp) reconnecting [ 4324.202379] Lustre: *** cfs_fail_loc=159, val=0*** [ 4324.214726] Lustre: Skipped 1 previous similar message [ 4339.835664] Lustre: lustre-MDT0001: Client 65870942-8f79-4461-af4e-0dbeb4f604ef (at 192.168.201.14@tcp) reconnecting [ 4342.145519] Lustre: *** cfs_fail_loc=159, val=0*** [ 4342.154288] Lustre: Skipped 2 previous similar messages [ 4357.818389] Lustre: lustre-MDT0001: Client 65870942-8f79-4461-af4e-0dbeb4f604ef (at 192.168.201.14@tcp) reconnecting [ 4364.435928] Lustre: 139159:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4366.838032] Lustre: *** cfs_fail_loc=159, val=0*** [ 4366.846114] Lustre: Skipped 5 previous similar messages [ 4383.438074] Lustre: lustre-MDT0000: Client 5b738b32-60ee-4091-bfca-490cbbc23d11 (at 192.168.201.14@tcp) reconnecting [ 4385.894690] Lustre: *** cfs_fail_loc=159, val=0*** [ 4385.901818] Lustre: Skipped 13 previous similar messages [ 4402.359184] Lustre: lustre-MDT0000: Client 5b738b32-60ee-4091-bfca-490cbbc23d11 (at 192.168.201.14@tcp) reconnecting [ 4406.907636] Lustre: 139451:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4418.529992] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4418.541646] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4418.553637] LustreError: Skipped 1 previous similar message [ 4418.592313] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4420.085498] Lustre: server umount lustre-OST0000 complete [ 4427.748340] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4427.756376] LustreError: Skipped 7 previous similar messages [ 4427.764384] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4427.780876] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4430.820798] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4430.820856] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4434.277447] Lustre: server umount lustre-MDT0000 complete [ 4435.966284] LustreError: 136117:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4436.026230] LustreError: 136117:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4436.960308] Lustre: 135429:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785313971/real 1785313971] req@0000000098e31822 x1872037138226048/t0(0) o400->lustre-OST0000-osc-MDT0001@0@lo:28/4 lens 224/224 e 0 to 1 dl 1785313987 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 4437.022410] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4441.521172] LustreError: 139399:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785313992 with bad export cookie 9836675742857044298 [ 4441.530520] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4441.546294] LustreError: 139399:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4442.112465] Lustre: server umount lustre-MDT0001 complete [ 4467.158961] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing unload_modules_local [ 4474.114365] Key type lgssc unregistered [ 4474.729546] LNet: 140526:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4474.751928] LNetError: 140526:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4475.821225] LNet: Removed LNI 192.168.201.114@tcp [ 4477.734584] Key type .llcrypt unregistered [ 4477.739382] Key type ._llcrypt unregistered [ 4514.084397] Key type ._llcrypt registered [ 4514.088688] Key type .llcrypt registered [ 4514.231523] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 4516.390746] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4516.430930] alg: No test for adler32 (adler32-zlib) [ 4517.665714] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [ 4518.186584] LNet: Added LNI 192.168.201.114@tcp [8/256/0/180] [ 4519.968326] Key type lgssc registered [ 4522.381356] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4532.946160] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4532.966374] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4534.477126] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4540.678492] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 4545.007610] LustreError: 141949:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4545.037180] LustreError: 141949:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4550.115148] LustreError: 141950:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4551.277061] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4551.827636] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4557.659936] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 4567.078464] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4575.693165] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4584.900339] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4585.336850] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4593.388789] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 4596.599435] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4601.115712] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4604.804497] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4611.850377] Lustre: 144536:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4614.945776] Lustre: 144585:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4616.145680] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4617.068799] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4621.024179] Lustre: 144731:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4621.816799] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4622.740755] Lustre: 144839:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4627.425351] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4627.429579] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4627.437876] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4630.511900] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4630.517137] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4630.531875] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4635.619632] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4635.646262] Lustre: Skipped 2 previous similar messages [ 4640.738607] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4640.755298] Lustre: Skipped 1 previous similar message [ 4641.248607] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4641.453323] Lustre: server umount lustre-OST0000 complete [ 4648.929521] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4648.942696] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4648.974934] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4650.986526] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4654.049132] Lustre: server umount lustre-MDT0000 complete [ 4656.104494] LustreError: 141949:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4656.145980] LustreError: 141949:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4660.504883] LustreError: 141929:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785314211 with bad export cookie 2595359421453721793 [ 4660.506181] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4660.529474] LustreError: 141929:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4661.167547] Lustre: server umount lustre-MDT0001 complete [ 4677.568644] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing unload_modules_local [ 4681.866815] Key type lgssc unregistered [ 4682.472358] LNet: 145893:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4682.481739] LNetError: 145893:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4682.530692] LNet: Removed LNI 192.168.201.114@tcp [ 4684.648600] Key type .llcrypt unregistered [ 4684.650025] Key type ._llcrypt unregistered [ 4714.417260] Key type ._llcrypt registered [ 4714.421308] Key type .llcrypt registered [ 4714.706709] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 4716.489046] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4716.571298] alg: No test for adler32 (adler32-zlib) [ 4717.634743] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [ 4718.000519] LNet: Added LNI 192.168.201.114@tcp [8/256/0/180] [ 4719.769423] Key type lgssc registered [ 4721.156200] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4731.509905] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4731.558747] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4733.049843] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4738.257800] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 4742.629315] LustreError: 147309:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4742.660186] LustreError: 147309:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4747.757263] LustreError: 147310:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4750.188247] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4750.661829] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4755.882756] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 4765.611950] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4772.779651] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4781.047375] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4781.607685] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4788.797703] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 4795.974733] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4796.463135] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4804.170131] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4813.600311] Lustre: *** cfs_fail_loc=159, val=0*** [ 4829.901218] Lustre: lustre-MDT0000: Client 7b6cd032-7f19-4037-9262-917d2b38ffac (at 192.168.201.14@tcp) reconnecting [ 4850.162865] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4850.180796] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4850.189428] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4854.073952] Lustre: server umount lustre-OST0000 complete [ 4860.392205] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4860.399828] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4860.418411] Lustre: Skipped 2 previous similar messages [ 4860.440291] Lustre: Skipped 2 previous similar messages [ 4865.169173] Lustre: server umount lustre-MDT0000 complete [ 4865.505532] LustreError: 147309:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4865.522858] LustreError: 147309:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4869.603255] LustreError: 148301:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4869.630910] LustreError: 148301:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4870.326886] LustreError: 147291:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785314420 with bad export cookie 4810798370112031278 [ 4870.334160] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4870.344259] LustreError: 147291:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4871.021545] Lustre: server umount lustre-MDT0001 complete [ 4884.295581] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing unload_modules_local [ 4889.051956] Key type lgssc unregistered [ 4889.354036] LNet: 151005:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4889.365318] LNetError: 151005:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4889.385053] LNet: Removed LNI 192.168.201.114@tcp [ 4890.329078] Key type .llcrypt unregistered [ 4890.330976] Key type ._llcrypt unregistered [ 4918.139364] Key type ._llcrypt registered [ 4918.141611] Key type .llcrypt registered [ 4918.275474] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 4920.239793] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4920.272244] alg: No test for adler32 (adler32-zlib) [ 4921.462666] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [ 4921.806638] LNet: Added LNI 192.168.201.114@tcp [8/256/0/180] [ 4923.552180] Key type lgssc registered [ 4924.844632] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4935.907910] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4935.971806] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4937.701867] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4944.140329] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 4947.941622] LustreError: 152422:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4947.959381] LustreError: 152422:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4953.066913] LustreError: 152423:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4956.524747] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4957.122289] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4962.899421] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 4972.186328] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4979.296543] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4986.619658] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4987.055441] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4994.491000] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 5002.433105] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 5004.331172] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5007.544808] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 5029.198941] Lustre: lustre-MDT0001: evicting fe7909df-f15e-4ad8-ad90-343fb165d72b (at 192.168.201.14@tcp) by administrative request [ 5029.213173] Lustre: Skipped 1 previous similar message [ 5059.041401] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5059.046852] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5059.057035] LustreError: Skipped 1 previous similar message [ 5059.058813] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5059.093146] Lustre: Skipped 1 previous similar message [ 5064.167818] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5064.175614] Lustre: Skipped 1 previous similar message [ 5064.379886] Lustre: server umount lustre-OST0000 complete [ 5069.284510] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5069.306246] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5069.310781] Lustre: Skipped 1 previous similar message [ 5069.335539] Lustre: Skipped 2 previous similar messages [ 5074.403687] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5075.568560] Lustre: server umount lustre-MDT0000 complete [ 5079.528598] LustreError: 152419:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5079.564863] LustreError: 152419:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5080.987050] LustreError: 154276:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785314631 with bad export cookie 6300773325834443331 [ 5080.988054] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5081.002695] LustreError: 154276:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5082.000275] Lustre: server umount lustre-MDT0001 complete [ 5096.860699] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing unload_modules_local [ 5102.382150] Key type lgssc unregistered [ 5103.028626] LNet: 156504:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5103.037430] LNetError: 156504:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5103.048438] LNet: Removed LNI 192.168.201.114@tcp [ 5104.232630] Key type .llcrypt unregistered [ 5104.239197] Key type ._llcrypt unregistered [ 5218.127753] Key type ._llcrypt registered [ 5218.129661] Key type .llcrypt registered [ 5218.321857] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 5220.319376] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5220.360381] alg: No test for adler32 (adler32-zlib) [ 5221.877674] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [ 5222.136410] LNet: Added LNI 192.168.201.114@tcp [8/256/0/180] [ 5223.792188] Key type lgssc registered [ 5225.281255] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5235.583875] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5235.599036] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5237.161821] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5241.824273] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 5247.473490] LustreError: 159001:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5247.493557] LustreError: 159001:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5252.583396] LustreError: 159002:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5253.020645] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5253.561368] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5260.321644] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 5270.263963] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5276.143954] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5284.157571] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5284.876075] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5293.840811] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 5298.258654] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5305.503565] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5308.498472] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 5323.745663] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5323.756225] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5323.764810] LustreError: Skipped 1 previous similar message [ 5323.771081] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5323.842285] Lustre: Skipped 1 previous similar message [ 5326.154283] Lustre: server umount lustre-OST0000 complete [ 5333.986577] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5334.002301] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5334.004432] Lustre: Skipped 1 previous similar message [ 5339.063225] Lustre: server umount lustre-MDT0000 complete [ 5339.125210] LustreError: 159733:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5339.173083] LustreError: 159733:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5344.242478] LustreError: 159001:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5344.300470] LustreError: 159001:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5346.768881] LustreError: 158981:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785314897 with bad export cookie 5766908673440947105 [ 5346.770872] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5346.780601] LustreError: 158981:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5347.297139] Lustre: server umount lustre-MDT0001 complete [ 5363.948894] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing unload_modules_local [ 5367.930947] Key type lgssc unregistered [ 5368.444419] LNet: 162565:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5368.457394] LNetError: 162565:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5368.504494] LNet: Removed LNI 192.168.201.114@tcp [ 5369.747771] Key type .llcrypt unregistered [ 5369.749470] Key type ._llcrypt unregistered [ 5405.947212] Key type ._llcrypt registered [ 5405.951237] Key type .llcrypt registered [ 5406.134419] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 5407.966645] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5408.007667] alg: No test for adler32 (adler32-zlib) [ 5409.383859] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [ 5409.844689] LNet: Added LNI 192.168.201.114@tcp [8/256/0/180] [ 5411.648280] Key type lgssc registered [ 5413.304880] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5431.473147] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 04:49:39 (1785314979) [ 5453.191667] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_hostid [ 5468.600464] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 5509.024144] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5509.046440] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5510.395191] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5510.453613] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5510.543591] Lustre: lustre-MDT0000: new disk, initializing [ 5510.670634] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5510.697838] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5516.429026] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 5528.499486] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5528.692328] Lustre: 166831:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5528.832381] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5528.841164] Lustre: Skipped 1 previous similar message [ 5528.967229] Lustre: lustre-MDT0001: new disk, initializing [ 5529.105050] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5529.148904] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5529.182711] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5535.603110] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 5545.765390] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5554.091212] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5563.794600] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5564.331699] Lustre: lustre-OST0000: new disk, initializing [ 5564.339404] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5564.348690] Lustre: 168799:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5564.495201] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5572.198754] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5572.207316] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5572.343871] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5572.797723] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 5583.599240] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5598.179457] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5598.187276] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5598.201067] LustreError: Skipped 1 previous similar message [ 5598.232794] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5598.255780] Lustre: Skipped 1 previous similar message [ 5601.437844] Lustre: server umount lustre-OST0000 complete [ 5605.857632] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5605.876535] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5605.894271] Lustre: Skipped 1 previous similar message [ 5605.905570] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5608.424242] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5608.427487] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5611.802726] Lustre: server umount lustre-MDT0000 complete [ 5615.753661] LustreError: 166824:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785315166 with bad export cookie 17873442125752235372 [ 5615.762454] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5615.766636] LustreError: 166824:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5616.057953] Lustre: server umount lustre-MDT0001 complete [ 5627.227816] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing unload_modules_local [ 5630.821183] Key type lgssc unregistered [ 5631.296436] LNet: 170707:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5631.306766] LNetError: 170707:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5631.336215] LNet: Removed LNI 192.168.201.114@tcp [ 5632.687304] Key type .llcrypt unregistered [ 5632.690941] Key type ._llcrypt unregistered [ 5658.686508] Key type ._llcrypt registered [ 5658.690892] Key type .llcrypt registered [ 5658.824394] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 5659.919844] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5660.107287] alg: No test for adler32 (adler32-zlib) [ 5661.197871] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [ 5661.380919] LNet: Added LNI 192.168.201.114@tcp [8/256/0/180] [ 5663.024165] Key type lgssc registered [ 5664.909038] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5673.019508] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5673.062414] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5674.862514] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5678.762940] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 5685.218987] LustreError: 172120:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5685.233286] LustreError: 172120:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5688.069866] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5688.288982] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5693.488465] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 5702.226206] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5707.978849] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5716.834987] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5717.214546] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5720.313374] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5723.493778] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 5732.308524] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5738.287901] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5738.305133] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5738.351431] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5738.379212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.404771] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5739.465194] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5739.487311] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5739.505623] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5739.523481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.534742] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5740.481417] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5740.496452] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5740.502943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.187231] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5741.190853] Lustre: Skipped 1 previous similar message [ 5741.193993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.202937] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5741.208155] Lustre: Skipped 1 previous similar message [ 5742.089051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.062955] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5743.073618] Lustre: Skipped 2 previous similar messages [ 5743.095378] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5743.110361] Lustre: Skipped 2 previous similar messages [ 5743.121935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.123896] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5744.137494] Lustre: Skipped 2 previous similar messages [ 5744.144386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.161228] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5744.170402] Lustre: Skipped 2 previous similar messages [ 5745.326479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.446721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.417772] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5747.421436] Lustre: Skipped 3 previous similar messages [ 5747.431958] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5747.438060] Lustre: Skipped 3 previous similar messages [ 5747.440834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.365445] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5748.373131] Lustre: Skipped 3 previous similar messages [ 5748.375499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.384634] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5748.390124] Lustre: Skipped 3 previous similar messages [ 5749.513032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.585607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.753261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.839568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.846831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.834769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.946286] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5755.952712] Lustre: Skipped 7 previous similar messages [ 5755.988856] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5756.002785] Lustre: Skipped 7 previous similar messages [ 5756.009788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.871380] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5756.880875] Lustre: Skipped 7 previous similar messages [ 5756.886322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.896662] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5756.913708] Lustre: Skipped 7 previous similar messages [ 5757.876665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.661694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.463883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.325114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.144328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.175616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.318197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.286986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.441697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.321416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.115277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.083679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.217847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.338420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.189394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.228129] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5772.235109] Lustre: Skipped 16 previous similar messages [ 5772.245435] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5772.254845] Lustre: Skipped 16 previous similar messages [ 5772.260776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.437941] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5773.442483] Lustre: Skipped 16 previous similar messages [ 5773.448749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.462597] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5773.470996] Lustre: Skipped 16 previous similar messages [ 5774.776565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.009540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.177595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.262988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.216882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.277034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.113650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.073968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.014340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.978951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.954809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.065670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.956455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.915324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.188518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.442543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.714468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.000202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.177996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.283261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.172222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.086257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.302404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.354126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.154422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.296378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.153040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.176328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.078556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.630142] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5805.641121] Lustre: Skipped 30 previous similar messages [ 5805.668904] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5805.676586] Lustre: Skipped 30 previous similar messages [ 5805.680197] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5805.682582] Lustre: Skipped 29 previous similar messages [ 5805.685512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.693471] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5805.696319] Lustre: Skipped 29 previous similar messages [ 5807.153325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.362057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.973816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.434810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.287098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.572624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.601787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.813197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.549876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.693971] LustreError: 172141:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5817.720854] LustreError: 172141:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5817.728920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.646154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.778163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.784768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.893986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.860122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.725015] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.904387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.597837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.512464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.465415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.595229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.571112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.759900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.638333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.866290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.813480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.579419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.366934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.432982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.522534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.457659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.539606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.862740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.347264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.513526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.526100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.482548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.664389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.629547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.936336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.003231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.025605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.555424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.228315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.982084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.693380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.312109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.941194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.930489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.180383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.659837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.302436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.664875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.214405] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5870.220922] Lustre: Skipped 53 previous similar messages [ 5870.247854] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5870.258432] Lustre: Skipped 53 previous similar messages [ 5870.266107] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5870.273818] Lustre: Skipped 53 previous similar messages [ 5870.276903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.291786] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5870.309798] Lustre: Skipped 53 previous similar messages [ 5871.711913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.060465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.576862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.986542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.160954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.263804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.398044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.516780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.439811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.701726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.128525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.206520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.643776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.627077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.635966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.798350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.298360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.807210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.362026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.664123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.030386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.000463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.014709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.009190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.142703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.177122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.230530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.281399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.498813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.682636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.932029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.053290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.334977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.681454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.560722] LustreError: 172141:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5912.569180] LustreError: 172141:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5912.584763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.406480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.411763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.300323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.452491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.443608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.316679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.273399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.274299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.296093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.479420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.565754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.838129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.223358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.515444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.534846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.456496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.555044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.377412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.497846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.559500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.639933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.605461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.604348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.707803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.859599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.968634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.900571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.856569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.588928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.037482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.167934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.509363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.628867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.900261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.639286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.883975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.980360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.087849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.686269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.311600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.471243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.876917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.007813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.404086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.204320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.482212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.868521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.243354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.725588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.796706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.073913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.209665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.345337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.694506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.245334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.423050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.241060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.998873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.619210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.239759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.601919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.090872] LustreError: 172141:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5987.107252] LustreError: 172141:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5987.116867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.561948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.504065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.781747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.801599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.511285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.423310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.554664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.959173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.307727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.705870] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5998.724259] Lustre: Skipped 106 previous similar messages [ 5998.749275] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5998.768580] Lustre: Skipped 106 previous similar messages [ 5998.783410] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5998.788576] Lustre: Skipped 106 previous similar messages [ 5998.812243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.840321] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5998.858153] Lustre: Skipped 106 previous similar messages [ 5999.993376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.326684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.673768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.646936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.690047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.854604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.871911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.834390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.097780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.341730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.275043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.041060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.916967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.802310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.738993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.615507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.385878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.523411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.609484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.557750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.453278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.574319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.809207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.993162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.959374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.122654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.423558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.309311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.439776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.804938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.879297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.889080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.922064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.013285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.942104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.194103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.321614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.516467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.649976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.553653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.734270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.202925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.550448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.870270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.804286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.802300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.198082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.429351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.990688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.297767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.198257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.148863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.827504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.245167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.625442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.003591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.736590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.016902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.907933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.819669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.635593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.623715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.586606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.825200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.099651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.271178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.454161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.406070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.665530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.944867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.258541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.596341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.001420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.234964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.413088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.676310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.693784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.657444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.793259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.047031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.000511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.057088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.215524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.194536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.218305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.240982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.500224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.711984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.999210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.039285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.198702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.657001] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.971212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.110742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.322169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.412610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.375277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.229771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.016232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.843754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.903114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.653984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.896750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.179783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.511274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.517610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.536895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.560827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.513334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.464230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.521290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.428362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.519413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.715335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.679276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.841709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.846700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.250695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.029082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.905302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.598577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.814404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.177423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.619831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.952690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.165390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.386519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.546713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.522538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.358492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.156542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.084673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.158859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.953164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.110163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.247924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.691244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.200570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.799848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.947350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.046854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.951622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.222178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.063460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.408094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.888604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.387674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.618136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.675869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.826092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.100590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.256165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.519698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.619319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.058387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.388401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.516789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.550982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.405403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.794609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.064415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.127688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.193918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.883957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.774817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.838888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.944116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.201220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.760026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.105555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.213686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.502404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.608361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.844163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.859521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.989390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.044168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.963030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.889920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.914082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.768593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.068609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.704962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.237857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.646424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.052331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.462472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.893986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.342128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.806644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.008827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.821991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.848917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.912884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.302458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.381142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.563413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.193484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.189023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.190724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.348910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.524546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.757469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.560137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.443555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.286511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.280147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.441165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.561213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.585481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.476412] LustreError: 172141:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6247.485556] LustreError: 172141:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6247.503622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.186286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.010855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.109950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.103050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.086720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.517195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.487455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.507462] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6255.515653] Lustre: Skipped 218 previous similar messages [ 6255.534486] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6255.549692] Lustre: Skipped 218 previous similar messages [ 6255.557119] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6255.562675] Lustre: Skipped 218 previous similar messages [ 6255.566571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.576116] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6255.580775] Lustre: Skipped 218 previous similar messages [ 6256.559084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.555737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.372891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.367388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.451033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.483255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.555386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.471080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.428142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.493051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.461512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.725835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.582173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.529037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.615229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.002758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.126132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.104303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.961916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.866689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.607784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.261064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.297320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.353327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.654095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.094205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.433954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.653945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.839260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.983221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.567261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.909458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.903353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.094911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.965684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.276474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.394179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.293993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.750716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.836869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.113906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.019634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.982139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.808109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.546865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.541566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.783435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.956457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.857638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.042963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.254461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.442415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.473892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.330373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.712206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.099278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.161796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.416533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6319.875665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.006081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.102879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.278474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.179907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.409933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.651267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.864285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.295494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.482235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6331.739941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.005395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.580574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.220705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.474943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6338.929191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.320178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.687806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6342.755482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6343.910130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.729376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.459903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.182207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.904724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6348.011938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.051752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.205056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6351.362423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.756994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.797233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6354.842426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.985932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.022223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.868218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.741591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.679454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.473679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6361.487212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6362.572451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.533742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.759870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.937572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.930445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.146416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.163929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.193437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.425883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.625212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6373.890580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.909790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.975835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.800305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6377.848566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.843958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.029787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.104201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6382.125389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.521403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6384.805971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6386.057425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.950163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6389.561363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.441229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6392.841582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.952387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6395.222328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6396.727854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.480163] LustreError: 172141:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6398.501006] LustreError: 172141:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6398.560318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.364560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.310956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6403.980334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.496523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6407.000151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6409.075664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.274794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.201705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6412.303411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.688189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.198430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.420789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.520131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6418.659281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.862649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6421.133676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6422.360711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6423.325710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.780206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6426.210375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6427.318941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6427.923928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6428.691592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6429.417838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.256957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.041652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6432.052108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6433.256125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6434.309875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.512767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6436.497168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6437.688489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.906473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6440.026290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.181221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.287469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6443.488571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6444.547216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.729237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6446.616481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6447.710210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6449.055336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6450.241342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.344494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6452.691708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6453.675464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6454.690244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.536971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6456.563611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6457.759250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6458.852959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6459.872906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6461.050379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6462.141085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6463.153717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6464.230105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6465.268540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6466.395034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6467.379902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6468.308909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6469.164801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6470.444565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6471.552577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6472.955758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6474.012509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6475.060637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6476.251843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6477.529603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6478.518654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6479.511379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6480.543272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6481.599628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6482.583952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6483.436338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6484.245404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6485.273851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6486.201464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6487.377527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6488.468766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6489.692535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6490.977352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6491.953158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6493.051306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6494.209340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6495.215670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6496.218487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6497.305757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6498.290055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6499.131439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6500.130393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6501.084947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6501.992368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6502.981211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6503.749300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6504.803283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6505.973172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6507.069522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6508.197514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6509.361603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6510.348388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6511.580077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6512.707866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6513.616423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6514.856281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6516.245696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6517.577973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6518.617905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6519.589278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6520.599612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6521.689479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6522.685833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6523.682543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6524.444292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6525.398123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6526.336195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6527.330982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6528.137966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6529.144400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6530.194732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6531.057368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6532.205302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6533.329889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6534.332440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6535.539119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6536.638510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6537.858621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6539.222165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6540.433487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6541.703626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6542.707537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6543.931811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6544.884662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6546.075242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6547.203596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6548.054065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6549.232497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6550.023483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6551.114629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6552.599387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6553.705910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6554.587586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6556.140548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6557.890340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6559.754283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6560.856387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6561.583512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6562.684930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6563.878100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6565.364164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6566.796609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6567.984654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6569.033660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6570.483221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6571.693344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6573.262112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6574.941628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6576.347268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6577.227491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6578.401157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6579.412399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6580.406846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6581.203677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6582.401214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6583.142947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6584.005294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6584.968977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6585.912792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6586.742518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6587.648551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6588.487475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6589.153093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6589.788852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6590.716567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6591.778288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6593.046717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6594.070806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6595.035361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6596.006773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6597.127513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6598.420616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6599.150900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6600.225262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6601.084949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6601.969724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6602.975851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6604.213378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6605.318686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6606.366300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6607.561947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6608.691627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6609.914117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6610.802149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6611.859332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6613.023703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6614.204443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6615.126241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6616.212538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6617.133875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6617.948622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6618.733410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6619.608257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6620.548775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6621.954812] LustreError: 172141:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6621.976172] LustreError: 172141:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6621.982043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6622.885157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6623.937179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6625.000242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6626.134450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6627.045680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6628.143989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6628.894669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6629.779658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6630.788462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6631.686215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6632.797306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6633.776479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6634.885786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6635.694075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6636.963872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6638.412589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6639.530168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6640.350982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6641.514751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6642.557102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6643.542061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6644.278603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6645.141422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6646.140492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6647.164843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6648.077709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6649.214280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6650.496447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6651.624890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6652.615827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6653.857489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6655.001560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6656.026034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6656.927899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6658.011450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6659.217706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6660.191406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6661.160798] LustreError: 172141:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6661.183820] LustreError: 172141:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6661.189884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6661.993470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6662.927763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6663.875595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6665.065124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6666.161167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6667.339749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6668.266160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6669.216377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6669.937959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6670.782451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6671.736445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6672.638269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6673.616468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6674.924563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6676.233789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6677.750417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6679.115881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6680.863299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6682.256528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6683.512110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6684.746294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6685.749031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6686.878672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6688.065268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6689.057431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6690.240148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6691.152053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6692.494844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6693.561044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6694.685969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6695.843422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6696.987959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6698.242775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6699.231871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6700.221810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6701.242663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6702.237695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6703.356913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6704.528660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6706.102199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6707.490284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6708.677651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6709.949080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6711.307609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6712.568169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6713.697207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6714.546338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6715.499795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6716.423067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6717.543590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6718.251949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6719.319051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6720.268388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6721.533205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6722.946134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6724.067563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6724.952687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6725.668336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6726.528960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6727.707833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6728.888496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6730.078256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6731.406210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6732.877067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6733.697779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6734.509286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6735.477169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6737.083574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6738.175020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6739.222869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6740.193213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6741.311367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6742.304706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6743.318949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6744.606865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6745.445413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6746.340908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6747.360718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6748.548958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6749.390337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6750.549282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6751.891846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6752.977128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6754.126987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6755.299595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6756.411979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6757.647721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6758.831794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6760.398502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6761.286225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6761.934969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6762.518902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6763.687256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6764.844303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6765.859385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6766.939295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6768.133889] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6768.148553] Lustre: Skipped 463 previous similar messages [ 6768.177850] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6768.215387] Lustre: Skipped 463 previous similar messages [ 6768.227719] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6768.231395] Lustre: Skipped 463 previous similar messages [ 6768.243346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6768.261466] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6768.273295] Lustre: Skipped 463 previous similar messages [ 6769.184519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6769.937848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6770.943418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6771.844747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6772.663435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6773.586902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6774.564219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6775.496230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6776.460927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6777.293689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6778.176954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6778.949812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6779.889592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6780.684469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6781.713352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6782.599860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6783.495871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6784.484983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6785.397758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6786.226971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6787.431762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6788.502914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6789.310867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6790.250504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6791.295146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6792.360053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6793.377276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6794.763484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6795.877451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6796.933115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6798.251284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6799.589723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6800.718277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6801.883452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6802.758880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6803.739162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6804.850329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6805.734888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6807.152049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6808.145555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6809.398872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6810.632687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6811.952071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6813.377033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6814.438305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6815.685120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6817.289782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6818.754215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6820.365517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6821.736894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6823.337415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6824.787329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6825.700867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6826.862647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6828.556444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6830.497508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6832.350674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6833.821104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6835.231552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6836.442828] LustreError: 172141:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6836.463707] LustreError: 172141:0:(osp_precreate.c:655:osp_precreate_send()) Skipped 2 previous similar messages [ 6836.479849] LustreError: 172141:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6836.488312] LustreError: 172141:0:(osp_precreate.c:1359:osp_precreate_thread()) Skipped 2 previous similar messages [ 6836.493263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6837.497471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6838.617253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6840.138117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6841.547024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6843.231312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6844.688544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6845.688339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6846.558805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6847.228393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6848.542692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6849.522659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6850.826516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6852.215424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6853.857993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6855.517912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6856.293502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6857.367794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6858.872170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6860.455793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6861.792026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6863.087896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6864.227151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6865.143867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6866.233648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6867.398969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6868.410352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6869.329495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6870.377952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6871.279439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6872.381282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7269.150972] LustreError: 173944:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 7269.162056] LustreError: 172141:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7269.169975] LustreError: 172141:0:(osp_precreate.c:655:osp_precreate_send()) Skipped 1 previous similar message [ 7924.203439] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7924.222914] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7924.224232] LustreError: Skipped 1 previous similar message [ 7924.248976] Lustre: Skipped 91 previous similar messages [ 7924.263654] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7925.050114] Lustre: server umount lustre-OST0000 complete [ 7930.344174] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7930.350114] Lustre: Skipped 2 previous similar messages [ 7935.469307] LustreError: 172120:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7935.718347] Lustre: server umount lustre-MDT0000 complete [ 7939.564388] LustreError: 172121:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7939.587559] LustreError: 172121:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7939.664964] LustreError: 172101:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785317490 with bad export cookie 3632486043695055729 [ 7939.682646] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7940.077972] Lustre: server umount lustre-MDT0001 complete [ 7952.976226] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing unload_modules_local [ 7955.943538] Key type lgssc unregistered [ 7956.236386] LNet: 227414:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7956.242587] LNetError: 227414:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7956.271550] LNet: Removed LNI 192.168.201.114@tcp [ 7957.308102] Key type .llcrypt unregistered [ 7957.311621] Key type ._llcrypt unregistered [ 7982.512495] Key type ._llcrypt registered [ 7982.514613] Key type .llcrypt registered [ 7982.611827] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 7983.737649] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7983.751253] alg: No test for adler32 (adler32-zlib) [ 7985.014254] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [ 7985.327235] LNet: Added LNI 192.168.201.114@tcp [8/256/0/180] [ 7987.112449] Key type lgssc registered [ 7988.997152] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8001.383699] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8001.403837] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8003.282278] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8010.600095] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 8012.784567] LustreError: 228833:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8012.824556] LustreError: 228833:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 8017.910752] LustreError: 228832:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8020.622917] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8020.979084] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8028.016286] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 8037.224667] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8044.119254] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8052.556751] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8053.136509] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8061.808758] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 8069.318218] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8072.585949] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 8079.157573] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 8080.982884] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8080.992868] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 8081.002728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8081.013247] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 8081.021789] LustreError: 228852:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 8085.627281] Lustre: lustre-OST0000: Not available for connect from 192.168.201.14@tcp (stopping) [ 8087.010055] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8087.026230] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8087.045753] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8088.043990] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8088.058040] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8088.065211] Lustre: Skipped 1 previous similar message [ 8090.706978] Lustre: lustre-OST0000: Not available for connect from 192.168.201.14@tcp (stopping) [ 8095.838804] Lustre: lustre-OST0000: Not available for connect from 192.168.201.14@tcp (stopping) [ 8095.856860] Lustre: Skipped 2 previous similar messages [ 8097.250345] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8097.572605] Lustre: server umount lustre-OST0000 complete [ 8101.041513] LustreError: 228852:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 8134.357240] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8134.722897] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8142.827234] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 8142.908322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8142.956567] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 8148.095975] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8148.129320] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 8153.676754] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8156.155682] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 8162.846391] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8de847c8a000.ost_server_uuid 50 [ 8164.645932] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8de847c8a000.ost_server_uuid in FULL state after 0 sec [ 8169.953983] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8169.968572] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8170.032501] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8171.677967] Lustre: server umount lustre-OST0000 complete [ 8179.169902] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8179.196136] Lustre: Skipped 1 previous similar message [ 8181.904356] Lustre: server umount lustre-MDT0000 complete [ 8184.301064] LustreError: 228829:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8184.338770] LustreError: 228829:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8186.934467] LustreError: 228814:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785317737 with bad export cookie 15531213051428776107 [ 8186.938186] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8186.956025] LustreError: 228814:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8196.907379] Lustre: server umount lustre-MDT0001 complete [ 8209.860509] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing unload_modules_local [ 8213.047698] Key type lgssc unregistered [ 8213.378703] LNet: 234046:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8213.383829] LNetError: 234046:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8213.401511] LNet: Removed LNI 192.168.201.114@tcp [ 8214.854678] Key type .llcrypt unregistered [ 8214.857789] Key type ._llcrypt unregistered [ 8255.857888] Key type ._llcrypt registered [ 8255.861369] Key type .llcrypt registered [ 8255.940978] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 8257.013194] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8257.041712] alg: No test for adler32 (adler32-zlib) [ 8258.056466] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [ 8258.297303] LNet: Added LNI 192.168.201.114@tcp [8/256/0/180] [ 8259.968985] Key type lgssc registered [ 8261.330495] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8272.044108] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8272.082565] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8273.769477] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8280.028936] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 8284.140704] LustreError: 235776:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8284.179372] LustreError: 235776:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 8289.257218] LustreError: 235777:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8291.999555] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8292.322993] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8297.505356] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 8301.447483] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 8309.437477] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8309.918580] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8316.957793] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 8319.144526] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 8327.505127] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8327.956855] Lustre: lustre-OST0001: new disk, initializing [ 8327.964277] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8327.988352] Lustre: 238091:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8328.183080] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8334.049468] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8334.070562] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8334.250403] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8337.236442] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 8349.099890] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8355.352780] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 8374.251643] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8374.266341] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8374.300479] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8375.269057] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8375.273629] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8375.305989] Lustre: Skipped 2 previous similar messages [ 8375.337693] Lustre: Skipped 3 previous similar messages [ 8377.519465] Lustre: server umount lustre-MDT0000 complete [ 8380.392884] LustreError: 238716:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8380.419696] LustreError: 238716:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 8382.706150] LustreError: 235756:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785317933 with bad export cookie 16009980624338872951 [ 8382.713948] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8382.723345] LustreError: 235756:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8383.429066] Lustre: server umount lustre-MDT0001 complete [ 8398.032572] Lustre: server umount lustre-OST0000 complete [ 8402.407247] Lustre: 235041:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785317936/real 1785317936] req@ffff9f7163540380 x1872041405955200/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785317952 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8402.443357] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8403.261261] Lustre: server umount lustre-OST0001 complete [ 8426.931704] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 8440.315401] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8440.699680] LustreError: 241200:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8440.802949] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8445.920889] LustreError: 241201:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8446.558977] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 8456.164156] LustreError: 241200:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8456.187774] LustreError: 241200:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8457.439456] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8458.296452] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8465.428432] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 8470.206623] Lustre: 242342:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8479.588905] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8480.514287] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8483.570550] LustreError: 242698:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8483.605513] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [ 8485.638431] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 8489.348582] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 8499.955105] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8508.060920] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 8518.632872] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8523.408483] Lustre: 244213:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8540.133158] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8540.148190] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8540.165638] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8544.669173] Lustre: server umount mylustre-MDT0000 complete [ 8546.276128] LustreError: 242457:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8546.317337] LustreError: 242457:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 9 previous similar messages [ 8550.617895] LustreError: 241176:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785318101 with bad export cookie 16009980624338875821 [ 8550.630210] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8550.644364] LustreError: 241176:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8551.402971] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8551.449878] Lustre: Skipped 3 previous similar messages [ 8551.744782] Lustre: server umount mylustre-MDT0001 complete [ 8557.344640] Lustre: server umount mylustre-OST0000 complete [ 8560.918804] Lustre: server umount mylustre-OST0001 complete [ 8585.928611] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 8601.567368] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8602.245619] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8602.257947] Lustre: Skipped 1 previous similar message [ 8611.146050] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 8612.338709] LustreError: 246528:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8612.373156] LustreError: 246528:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 8622.616972] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8628.099465] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 8632.438729] Lustre: 247669:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8643.423206] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8647.382628] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 8648.475209] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:161) [ 8654.284668] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 8665.254536] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8670.729944] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:65) [ 8670.827935] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 8678.749991] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8683.002208] Lustre: 249543:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8694.753170] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8694.760537] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8694.767222] Lustre: Skipped 1 previous similar message [ 8694.771054] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 8694.774229] Lustre: Skipped 4 previous similar messages [ 8697.774294] Lustre: server umount tfs-MDT0000 complete [ 8704.254424] LustreError: 246503:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785318254 with bad export cookie 16009980624338877774 [ 8704.262461] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8704.282714] LustreError: 246503:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8705.192366] Lustre: server umount tfs-MDT0001 complete [ 8721.127121] Lustre: server umount tfs-OST0000 complete [ 8722.848204] Lustre: 235039:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785318257/real 1785318257] req@ffff9f7248d29c00 x1872041406085888/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785318273 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8722.883858] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8722.903822] Lustre: Skipped 3 previous similar messages [ 8725.521150] Lustre: server umount tfs-OST0001 complete [ 8750.959712] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 8765.446101] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8766.240860] LustreError: 251794:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8766.277223] LustreError: 251794:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 15 previous similar messages [ 8766.405141] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8766.409553] Lustre: Skipped 3 previous similar messages [ 8772.612418] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 8781.931923] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8789.512284] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 8793.413739] Lustre: 252937:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8800.886790] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8808.359662] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:193) [ 8809.589750] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 8811.493685] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 8823.739823] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8829.459944] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:97) [ 8833.205940] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 8843.375216] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8848.543129] Lustre: 254807:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8858.558659] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 05:46:46 (1785318406) [ 8874.979055] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8874.990985] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8875.009979] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8875.018946] Lustre: Skipped 4 previous similar messages [ 8885.730652] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8885.745368] Lustre: Skipped 10 previous similar messages [ 8889.313542] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8889.676392] Lustre: server umount lustre-MDT0000 complete [ 8894.422172] LustreError: 251769:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785318444 with bad export cookie 16009980624338879692 [ 8894.428192] LustreError: 251769:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8894.434559] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8901.032691] Lustre: server umount lustre-MDT0001 complete [ 8912.494631] Lustre: server umount lustre-OST0000 complete [ 8923.827437] Lustre: server umount lustre-OST0001 complete [ 8931.713087] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_hostid [ 8943.398417] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 8993.749883] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 8993.749883] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8993.749883] [ 8993.795604] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8994.085732] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8994.115994] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8994.280745] Lustre: lustre-MDT0000: new disk, initializing [ 8994.403536] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8994.406393] Lustre: Skipped 3 previous similar messages [ 8994.458773] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9001.778773] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 9016.910966] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 9016.910966] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 9016.910966] [ 9016.974522] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 9017.151515] Lustre: 257998:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9017.202593] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9017.206023] Lustre: Skipped 1 previous similar message [ 9017.273296] Lustre: lustre-MDT0001: new disk, initializing [ 9017.402794] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9017.419258] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9022.183274] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 9033.012033] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9033.193735] Lustre: lustre-OST0000: new disk, initializing [ 9033.196540] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9033.200908] Lustre: 259580:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9035.046268] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9035.062474] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9035.174119] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9039.254103] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 9048.155779] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9048.271830] Lustre: lustre-OST0001: new disk, initializing [ 9048.275239] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 9048.280561] Lustre: 260448:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9050.396729] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 9050.415344] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 9050.587561] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 9056.777978] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 9073.165942] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 05:50:21 (1785318621) [ 9093.632804] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 05:50:41 (1785318641) [ 9112.035854] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9112.058022] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9112.075983] Lustre: Skipped 6 previous similar messages [ 9112.078671] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9112.106280] Lustre: Skipped 5 previous similar messages [ 9115.818299] Lustre: server umount lustre-OST0000 complete [ 9117.169032] LustreError: 259698:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9117.187898] LustreError: 259698:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 19 previous similar messages [ 9127.108270] Lustre: server umount lustre-MDT0000 complete [ 9133.455332] LustreError: 257991:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785318684 with bad export cookie 16009980624338881470 [ 9133.480421] LustreError: 257991:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9133.485223] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9140.358205] Lustre: server umount lustre-MDT0001 complete [ 9154.676907] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing unload_modules_local [ 9167.671625] Lustre: server umount lustre-OST0001 complete [ 9175.675646] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_hostid [ 9195.939230] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 9240.727542] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9241.089292] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9241.160455] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9241.295758] Lustre: lustre-MDT0000: new disk, initializing [ 9241.526682] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9247.121557] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 9259.534724] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9259.769370] Lustre: 265007:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9259.980960] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9259.992742] Lustre: Skipped 1 previous similar message [ 9260.139776] Lustre: lustre-MDT0001: new disk, initializing [ 9260.340238] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9260.348791] Lustre: Skipped 4 previous similar messages [ 9260.392714] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9260.408468] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9269.911928] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 9285.055820] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9293.076142] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9301.532335] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9302.040848] Lustre: lustre-OST0000: new disk, initializing [ 9302.045732] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9302.055700] Lustre: 266982:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9304.028196] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9304.040297] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9304.201962] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9311.432402] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 9323.771622] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9334.759049] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9334.771915] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9334.781607] LustreError: Skipped 2 previous similar messages [ 9334.783490] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9334.783498] Lustre: Skipped 5 previous similar messages [ 9334.826342] Lustre: Skipped 5 previous similar messages [ 9340.019528] Lustre: server umount lustre-OST0000 complete [ 9351.064884] Lustre: server umount lustre-MDT0000 complete [ 9356.036034] LustreError: 265002:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785318906 with bad export cookie 16009980624338882940 [ 9356.049017] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9356.054691] LustreError: 265002:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 9356.525894] Lustre: server umount lustre-MDT0001 complete [ 9370.313849] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing unload_modules_local [ 9373.462311] Key type lgssc unregistered [ 9373.777852] LNet: 268743:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9373.784210] LNetError: 268743:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9373.801356] LNet: Removed LNI 192.168.201.114@tcp [ 9375.323240] Key type .llcrypt unregistered [ 9375.324831] Key type ._llcrypt unregistered [ 9411.834344] Key type ._llcrypt registered [ 9411.835897] Key type .llcrypt registered [ 9411.975109] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 9413.231509] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9413.336018] alg: No test for adler32 (adler32-zlib) [ 9414.818508] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [ 9415.105273] LNet: Added LNI 192.168.201.114@tcp [8/256/0/180] [ 9416.840303] Key type lgssc registered [ 9418.491354] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9427.157966] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9427.186956] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9428.740678] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9435.015838] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 9439.207382] LustreError: 270205:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9439.241847] LustreError: 270205:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 9444.321905] LustreError: 270206:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9445.933218] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9446.348715] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9451.840468] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 9462.461334] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9469.210774] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9478.165886] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9478.743467] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9487.142812] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 9498.942195] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9499.260687] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 9500.466753] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 9504.738108] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9504.749270] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9504.753172] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9504.762607] Lustre: Skipped 1 previous similar message [ 9504.780737] Lustre: Skipped 1 previous similar message [ 9505.414652] Lustre: server umount lustre-OST0000 complete [ 9512.930803] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9512.949241] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9512.974929] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9514.977663] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9514.993238] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9517.795674] Lustre: server umount lustre-MDT0000 complete [ 9520.106756] LustreError: 270202:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9520.135659] LustreError: 270202:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9525.031229] LustreError: 270185:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785319075 with bad export cookie 2693597110883480923 [ 9525.041687] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9525.044679] LustreError: 270185:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9525.373036] Lustre: server umount lustre-MDT0001 complete [ 9538.099709] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing unload_modules_local [ 9541.616962] Key type lgssc unregistered [ 9541.954026] LNet: 273797:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9541.962728] LNetError: 273797:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9541.981733] LNet: Removed LNI 192.168.201.114@tcp [ 9543.020696] Key type .llcrypt unregistered [ 9543.022048] Key type ._llcrypt unregistered [ 9568.909707] Key type ._llcrypt registered [ 9568.912715] Key type .llcrypt registered [ 9569.014815] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 9570.578804] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9570.697899] alg: No test for adler32 (adler32-zlib) [ 9571.992422] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [ 9572.388046] LNet: Added LNI 192.168.201.114@tcp [8/256/0/180] [ 9574.128395] Key type lgssc registered [ 9576.594267] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9591.692882] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9591.711550] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9607.201241] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 9612.320498] LustreError: 274995:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.114@tcp: failed processing log, type 4: rc = -110 [ 9639.009400] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9645.642346] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 9655.849979] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9659.146183] Lustre: server umount lustre-OST0000 complete [ 9684.183463] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing unload_modules_local [ 9686.958671] Key type lgssc unregistered [ 9687.339504] LNet: 276649:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9687.344997] LNetError: 276649:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9687.361349] LNet: Removed LNI 192.168.201.114@tcp [ 9688.080393] Key type .llcrypt unregistered [ 9688.083472] Key type ._llcrypt unregistered [ 9744.725147] Key type ._llcrypt registered [ 9744.730104] Key type .llcrypt registered [ 9744.908375] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 9746.231074] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9746.325866] alg: No test for adler32 (adler32-zlib) [ 9747.582134] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [ 9748.024016] LNet: Added LNI 192.168.201.114@tcp [8/256/0/180] [ 9749.888636] Key type lgssc registered [ 9751.753374] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9768.864358] loop0: detected capacity change from 0 to 204800000 [ 9769.151710] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9769.166498] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9769.785651] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9771.158551] loop0: detected capacity change from 0 to 204800000 [ 9771.410834] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9771.422963] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9771.892178] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9773.346314] loop0: detected capacity change from 0 to 204800000 [ 9773.530302] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9773.539373] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9773.635408] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9774.909624] loop0: detected capacity change from 0 to 204800000 [ 9775.181546] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9775.190987] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9775.290084] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9776.611878] loop0: detected capacity change from 0 to 204800000 [ 9776.663290] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9782.368298] loop0: detected capacity change from 0 to 204800000 [ 9782.420724] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9788.738959] loop0: detected capacity change from 0 to 204800000 [ 9788.784264] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9795.333630] loop0: detected capacity change from 0 to 204800000 [ 9795.381063] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9799.762527] loop0: detected capacity change from 0 to 204800000 [ 9799.878848] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9799.921987] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9799.974573] Lustre: 280096:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9799.983360] Lustre: 280096:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 9799.991208] Lustre: 280096:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 9799.999983] Lustre: 280096:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9800.004783] Lustre: 280096:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9800.011924] Lustre: 280096:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9800.016821] Lustre: 280096:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 9800.023174] Lustre: 280096:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 9800.033120] Lustre: 280096:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 9800.042437] Lustre: 280096:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 9800.052914] Lustre: 280096:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 9800.064361] Lustre: 280096:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 9800.077371] Lustre: 280096:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 9800.100163] Lustre: 280096:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 9808.488159] Lustre: server umount MGS complete [ 9810.042635] loop0: detected capacity change from 0 to 204800000 [ 9810.178184] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9811.729628] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9811.887906] Lustre: lustre-MDT0000: Aborting client recovery [ 9813.490269] loop1: detected capacity change from 0 to 204800000 [ 9813.725433] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9813.885488] Lustre: 280530:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9813.927158] Lustre: 280530:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 9813.959643] Lustre: 280530:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 9814.004238] Lustre: 280530:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9814.030233] Lustre: 280530:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9814.047681] Lustre: 280530:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 9814.067369] Lustre: 280530:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9814.131561] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9814.431593] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9814.536562] Lustre: lustre-MDT0001: Aborting client recovery [ 9815.830850] loop2: detected capacity change from 0 to 204800000 [ 9816.001257] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9816.123471] Lustre: 280608:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9816.133826] Lustre: 280608:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9816.148522] Lustre: 280608:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9816.157639] Lustre: 280608:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9816.165448] Lustre: 280608:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9816.173580] Lustre: 280608:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9816.180598] Lustre: 280608:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9816.328363] Lustre: Found index 0 for lustre-OST0000, updating log [ 9816.352866] Lustre: 280627:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9816.434649] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9816.452269] Lustre: lustre-OST0000: Aborting recovery [ 9817.278843] loop3: detected capacity change from 0 to 204800000 [ 9817.464808] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9817.630964] Lustre: 280692:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9817.664514] Lustre: 280692:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9817.678172] Lustre: 280692:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9817.703269] Lustre: 280692:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9817.715787] Lustre: 280692:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9817.728688] Lustre: 280692:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9817.743232] Lustre: 280692:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9817.772546] Lustre: Found index 1 for lustre-OST0001, updating log [ 9817.905084] Lustre: 280700:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9818.181737] LustreError: 280474:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 6, retries 0, failed: rc = -108 [ 9818.204055] Lustre: lustre-OST0001: Aborting recovery [ 9822.267243] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9822.275775] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9822.391444] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 9822.590953] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 9824.361237] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 9824.388541] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 9828.051080] LustreError: 280441:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 218us: rc = -2 [ 9828.577690] LustreError: 280442:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 484us: rc = -2 [ 9828.587448] LustreError: 280442:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 22 previous similar messages [ 9845.219808] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9845.221288] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9845.240801] Lustre: Skipped 2 previous similar messages [ 9845.260742] Lustre: Skipped 3 previous similar messages [ 9847.893240] Lustre: server umount lustre-MDT0000 complete [ 9849.122916] LustreError: 280425:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785319399 with bad export cookie 15105056846352280730 [ 9849.139363] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9850.339555] LustreError: 280983:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9850.347352] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9850.347963] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9850.370263] LustreError: 280983:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 9850.392265] Lustre: Skipped 2 previous similar messages [ 9854.369104] LustreError: 280440:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9854.369860] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9854.399430] LustreError: 280440:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9854.415474] Lustre: Skipped 2 previous similar messages [ 9855.937113] Lustre: server umount lustre-MDT0001 complete [ 9871.328133] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9871.812759] Lustre: server umount lustre-OST0000 complete [ 9879.981345] Lustre: server umount lustre-OST0001 complete [ 9889.631259] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 06:03:57 (1785319437) [ 9928.206466] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_hostid [ 9937.157315] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [ 9980.817446] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9981.041548] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9981.090750] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9981.203860] Lustre: lustre-MDT0000: new disk, initializing [ 9981.331783] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9981.345985] Lustre: Skipped 1 previous similar message [ 9981.369960] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9986.594230] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [ 9999.212798] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9999.360588] Lustre: 284770:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9999.415135] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9999.417795] Lustre: Skipped 1 previous similar message [ 9999.543089] Lustre: lustre-MDT0001: new disk, initializing [ 9999.628400] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9999.663288] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9999.668270] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10004.502848] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [10015.815450] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10023.916584] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10033.810736] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10034.397835] Lustre: lustre-OST0000: new disk, initializing [10034.417638] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10034.429465] Lustre: 286742:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10034.637509] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10041.466747] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10041.490377] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10041.708508] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10043.639466] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [10055.007051] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10060.453777] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10072.902465] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10079.449105] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10090.270205] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [10128.868370] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10128.869316] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10128.875248] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10128.894380] Lustre: Skipped 1 previous similar message [10131.699109] Lustre: server umount lustre-OST0000 complete [10138.083652] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10138.097558] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10138.163610] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10138.171951] Lustre: Skipped 1 previous similar message [10149.356371] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10149.377222] Lustre: Skipped 5 previous similar messages [10151.396395] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10151.794372] Lustre: server umount lustre-MDT0000 complete [10154.468399] LustreError: 285853:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10154.509835] LustreError: 285853:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10157.559638] LustreError: 284764:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785319708 with bad export cookie 15105056846352283460 [10157.583311] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10158.253260] Lustre: server umount lustre-MDT0001 complete [10168.996262] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10174.597717] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [10180.947629] Lustre: server umount MGS complete [10194.128628] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10194.466275] LustreError: 290612:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10194.496508] LustreError: 290625:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10194.565647] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10200.046995] LustreError: 290613:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10200.230971] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [10210.031037] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10210.273851] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [10215.666217] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [10223.568338] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10230.043873] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10237.430183] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10237.732288] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10237.742416] Lustre: Skipped 1 previous similar message [10245.222471] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [10254.510702] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10291.171306] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10291.202282] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10291.209104] Lustre: Skipped 1 previous similar message [10291.244572] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10291.258756] Lustre: Skipped 1 previous similar message [10296.684694] Lustre: server umount lustre-OST0000 complete [10302.948695] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10302.965778] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10302.991409] Lustre: Skipped 1 previous similar message [10317.281141] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10317.667884] Lustre: server umount lustre-MDT0000 complete [10320.866798] LustreError: 290608:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10320.884869] LustreError: 290608:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10321.431105] LustreError: 290594:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785319871 with bad export cookie 15105056846352285294 [10321.443689] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10321.446166] LustreError: 290594:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [10321.884504] Lustre: server umount lustre-MDT0001 complete [10334.531985] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing unload_modules_local [10337.626631] Key type lgssc unregistered [10338.040292] LNet: 294561:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10338.058442] LNetError: 294561:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10338.077517] LNet: Removed LNI 192.168.201.114@tcp [10339.217394] Key type .llcrypt unregistered [10339.220052] Key type ._llcrypt unregistered [10381.766303] Key type ._llcrypt registered [10381.768179] Key type .llcrypt registered [10381.866362] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_hostid [10397.704112] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [10398.994346] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10399.085782] alg: No test for adler32 (adler32-zlib) [10400.341195] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [10400.516533] LNet: Added LNI 192.168.201.114@tcp [8/256/0/180] [10402.152212] Key type lgssc registered [10403.427754] Lustre: Echo OBD driver; http://www.lustre.org/ [10444.681559] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10444.744394] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10446.312129] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10446.425337] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10446.517634] Lustre: lustre-MDT0000: new disk, initializing [10446.588035] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10446.618164] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10453.211859] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [10468.520517] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10468.735436] Lustre: 298527:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10468.807048] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10468.811522] Lustre: Skipped 1 previous similar message [10468.895128] Lustre: lustre-MDT0001: new disk, initializing [10469.060227] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10469.193690] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10469.223683] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10476.069831] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [10491.196954] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10500.517130] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10509.899461] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10510.576826] Lustre: lustre-OST0000: new disk, initializing [10510.581654] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10510.600860] Lustre: 300498:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10510.756883] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10519.168278] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10519.187708] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10519.361149] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10519.477081] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [10529.882434] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10537.518906] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10546.794749] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10551.068131] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10557.167268] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [10606.569570] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10606.571392] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10606.595291] LustreError: Skipped 1 previous similar message [10606.613646] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10606.635048] Lustre: Skipped 1 previous similar message [10611.685013] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10611.696536] Lustre: Skipped 2 previous similar messages [10616.288319] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10616.723142] Lustre: server umount lustre-OST0000 complete [10621.921093] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10621.922890] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10621.931239] Lustre: Skipped 1 previous similar message [10621.972929] Lustre: Skipped 1 previous similar message [10627.058651] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10627.067506] Lustre: Skipped 1 previous similar message [10632.169132] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10632.180186] Lustre: Skipped 1 previous similar message [10636.257985] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10636.874282] Lustre: server umount lustre-MDT0000 complete [10641.659567] LustreError: 298517:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785320192 with bad export cookie 10737865643949166927 [10641.666748] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10641.680298] LustreError: 298517:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10642.117886] Lustre: server umount lustre-MDT0001 complete [10651.870796] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10657.495376] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [10662.891848] Lustre: server umount MGS complete [10675.789480] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10676.607596] LustreError: 304267:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10676.632908] LustreError: 304267:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [10676.650082] LustreError: 304280:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10676.785903] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10681.848041] LustreError: 304268:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10682.314356] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [10686.947557] LustreError: 304267:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10692.104298] LustreError: 304268:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10693.312792] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10693.828115] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10700.611384] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [10710.595502] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10716.993910] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10726.064819] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10726.729357] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10734.597958] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [10741.838595] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10778.091421] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10778.094226] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10778.106131] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10784.341209] Lustre: server umount lustre-OST0000 complete [10790.911094] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10790.914665] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10790.923103] Lustre: Skipped 1 previous similar message [10795.118845] Lustre: server umount lustre-MDT0000 complete [10798.563939] LustreError: 304263:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10798.584042] LustreError: 304263:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10800.276874] LustreError: 304248:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785320350 with bad export cookie 10737865643949168782 [10800.278427] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10800.291066] LustreError: 304248:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10801.433587] Lustre: server umount lustre-MDT0001 complete [10814.991449] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing unload_modules_local [10818.059385] Key type lgssc unregistered [10818.465642] LNet: 308214:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10818.479861] LNetError: 308214:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10818.514796] LNet: Removed LNI 192.168.201.114@tcp [10819.668384] Key type .llcrypt unregistered [10819.670585] Key type ._llcrypt unregistered [10844.710941] Key type ._llcrypt registered [10844.712051] Key type .llcrypt registered [10844.895284] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_hostid [10862.111438] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [10863.738280] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10863.918251] alg: No test for adler32 (adler32-zlib) [10865.099805] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [10865.391738] LNet: Added LNI 192.168.201.114@tcp [8/256/0/180] [10867.128189] Key type lgssc registered [10868.279509] Lustre: Echo OBD driver; http://www.lustre.org/ [10902.980601] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [10924.379749] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10924.411265] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10925.801940] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10925.866895] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10925.980886] Lustre: lustre-MDT0000: new disk, initializing [10926.091844] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10926.107649] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10931.407291] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [10944.697967] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10945.059291] Lustre: lustre-OST0000: new disk, initializing [10945.064274] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10945.078621] Lustre: Skipped 1 previous similar message [10945.089605] Lustre: 312269:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10945.158790] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10951.659333] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [10952.860328] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [10952.876220] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [10952.898801] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [11901.522547] Lustre: server umount lustre-MDT0000 complete [11906.442032] LustreError: 311456:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785321456 with bad export cookie 3564893556740656792 [11906.459576] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11915.951308] Lustre: server umount lustre-OST0000 complete [11948.735644] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_hostid [11967.880459] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [12015.582891] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12015.929547] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12015.984210] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12016.087400] Lustre: lustre-MDT0000: new disk, initializing [12016.215853] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12016.261299] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12021.847686] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [12033.974542] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12034.104983] Lustre: 316650:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12034.156397] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12034.162909] Lustre: Skipped 1 previous similar message [12034.232414] Lustre: lustre-MDT0001: new disk, initializing [12034.350902] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12034.393625] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12034.411264] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12040.404070] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [12051.557735] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12058.846546] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12067.775372] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12068.095563] Lustre: lustre-OST0000: new disk, initializing [12068.104444] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12068.117946] Lustre: 318618:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12068.327873] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12075.141064] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12075.151920] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12075.332202] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12077.571893] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [12091.746788] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12098.020907] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12098.396698] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12105.859059] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [12106.199547] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12111.334864] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12111.344823] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12111.344924] LustreError: Skipped 1 previous similar message [12111.345781] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12111.411449] Lustre: Skipped 1 previous similar message [12115.672396] Lustre: server umount lustre-OST0000 complete [12121.570083] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12121.578348] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12121.592870] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12121.592878] Lustre: Skipped 1 previous similar message [12121.636291] Lustre: Skipped 1 previous similar message [12126.689093] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12126.697545] Lustre: Skipped 2 previous similar messages [12131.809912] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12131.825957] Lustre: Skipped 1 previous similar message [12135.392136] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12135.811536] Lustre: server umount lustre-MDT0000 complete [12140.861426] LustreError: 316641:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785321691 with bad export cookie 3564893556743597611 [12140.873188] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12140.886129] LustreError: 316641:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12141.423664] Lustre: server umount lustre-MDT0001 complete [12146.235229] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [12148.035451] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 06:41:36 (1785321696) [12156.626620] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12157.225393] LustreError: 320938:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12157.248783] LustreError: 320938:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [12157.315873] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12162.543666] LustreError: 320939:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12165.107319] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [12166.630611] LustreError: 320938:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12171.747160] LustreError: 320939:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12176.867113] LustreError: 320938:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12178.093836] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12178.493241] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12181.921314] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [12190.989270] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12198.592968] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12206.615551] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12207.244261] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12214.775586] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [12226.956667] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12237.440042] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12237.913875] Lustre: lustre-OST0001: new disk, initializing [12237.920930] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [12237.930584] Lustre: 323780:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12238.098983] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12243.108085] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [12243.126677] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [12243.201124] LustreError: 323962:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12244.268496] LustreError: 323962:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12244.283557] LustreError: 323962:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [12245.294353] LustreError: 323967:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [12247.315881] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [12247.411481] LustreError: 323967:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [12247.428489] LustreError: 323967:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [12251.559418] LustreError: 323967:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [12251.571806] LustreError: 323967:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [12259.196412] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12259.423163] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12259.883236] LustreError: 323967:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [12259.894667] LustreError: 323967:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [12266.036312] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8de87745c000.ost_server_uuid 50 [12268.303496] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8de87745c000.ost_server_uuid in FULL state after 0 sec [12276.456567] LustreError: 323967:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [12276.482752] LustreError: 323967:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 32 previous similar messages [12278.339878] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8de87745c000.ost_server_uuid 50 [12282.279358] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8de87745c000.ost_server_uuid in IDLE state after 0 sec [12284.823260] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [12304.875817] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12304.877026] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12304.895743] Lustre: Skipped 1 previous similar message [12309.314857] Lustre: server umount lustre-OST0001 complete [12310.011280] LustreError: 323545:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12310.038876] LustreError: 323545:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12319.203101] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12319.212524] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12319.229543] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12319.241402] Lustre: Skipped 2 previous similar messages [12322.882707] Lustre: server umount lustre-OST0000 complete [12330.484706] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12330.500497] Lustre: Skipped 2 previous similar messages [12334.088094] Lustre: server umount lustre-MDT0000 complete [12335.592326] LustreError: 320939:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12335.606754] LustreError: 320939:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [12338.739979] LustreError: 320920:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785321889 with bad export cookie 3564893556743598493 [12338.747144] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12338.751457] LustreError: 320920:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [12339.284935] Lustre: server umount lustre-MDT0001 complete [12352.104328] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing unload_modules_local [12356.010846] Key type lgssc unregistered [12356.254710] LNet: 326473:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12356.276874] LNetError: 326473:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12356.316086] LNet: Removed LNI 192.168.201.114@tcp [12357.566782] Key type .llcrypt unregistered [12357.569336] Key type ._llcrypt unregistered [12387.719323] Key type ._llcrypt registered [12387.721208] Key type .llcrypt registered [12387.824287] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [12389.477763] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12389.555666] alg: No test for adler32 (adler32-zlib) [12390.808471] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [12391.145106] LNet: Added LNI 192.168.201.114@tcp [8/256/0/180] [12392.840204] Key type lgssc registered [12394.572763] Lustre: Echo OBD driver; http://www.lustre.org/ [12404.434053] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12404.455621] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12406.136696] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12411.413229] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [12415.461699] LustreError: 327895:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12415.480971] LustreError: 327895:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12420.111816] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12420.585039] LustreError: 327896:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12425.709645] LustreError: 327895:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12427.800310] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12428.249443] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12433.339337] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [12444.498326] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12444.786323] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12450.934366] LustreError: 329576:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12450.947566] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [12452.494566] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [12458.470268] LustreError: 329852:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12458.490492] LustreError: 329852:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [12460.290690] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12467.935736] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12468.197476] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12473.362487] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [12475.435601] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [12483.321093] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12490.360613] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [12490.527296] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12594.148941] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12594.169292] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12594.181280] Lustre: Skipped 1 previous similar message [12594.191889] Lustre: Skipped 1 previous similar message [12598.569998] Lustre: server umount lustre-OST0001 complete [12599.276885] LustreError: 329852:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12599.354061] LustreError: 329852:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [12606.947626] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12606.954504] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12606.976553] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12609.509963] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12609.515518] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12612.274955] Lustre: server umount lustre-OST0000 complete [12617.699357] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12617.714295] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12617.736235] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12617.742725] Lustre: Skipped 1 previous similar message [12623.193647] Lustre: server umount lustre-MDT0000 complete [12624.873554] LustreError: 328818:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12624.900436] LustreError: 328818:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [12628.174978] LustreError: 331581:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785322178 with bad export cookie 15484459854328820831 [12628.192473] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12628.192957] LustreError: 331581:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12628.991064] Lustre: server umount lustre-MDT0001 complete [12641.939485] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing unload_modules_local [12644.985455] Key type lgssc unregistered [12645.320712] LNet: 332840:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12645.328744] LNetError: 332840:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12645.355245] LNet: Removed LNI 192.168.201.114@tcp [12646.664149] Key type .llcrypt unregistered [12646.665376] Key type ._llcrypt unregistered [12698.157274] Key type ._llcrypt registered [12698.158937] Key type .llcrypt registered [12698.256351] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [12699.277688] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12699.467154] alg: No test for adler32 (adler32-zlib) [12700.554536] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [12700.782620] LNet: Added LNI 192.168.201.114@tcp [8/256/0/180] [12702.497221] Key type lgssc registered [12703.794238] Lustre: Echo OBD driver; http://www.lustre.org/ [12715.923674] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12715.937940] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12717.611303] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12723.383295] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [12727.778427] LustreError: 335132:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12727.800246] LustreError: 335132:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12732.907136] LustreError: 335131:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12733.090669] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12733.460220] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12738.318262] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [12741.528400] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [12749.733285] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12750.312403] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12757.086614] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [12757.499914] LustreError: 336625:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12762.601886] LustreError: 336626:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12762.632385] LustreError: 336626:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12762.646681] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [12765.555406] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12765.804825] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12770.805544] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [12772.869395] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [12780.612268] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12788.225122] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [12801.524026] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12801.527975] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12801.544736] Lustre: Skipped 3 previous similar messages [12801.559791] Lustre: Skipped 3 previous similar messages [12806.628710] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12806.635059] Lustre: Skipped 3 previous similar messages [12807.157807] Lustre: server umount lustre-MDT0000 complete [12810.790617] LustreError: 335113:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785322361 with bad export cookie 1987978934765889777 [12810.799978] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12810.800576] LustreError: 335113:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12811.133721] Lustre: server umount lustre-MDT0001 complete [12825.430520] Lustre: server umount lustre-OST0000 complete [12828.128098] Lustre: 334395:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785322362/real 1785322362] req@ffff9f727ad3d880 x1872046064728960/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785322378 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [12828.176959] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12829.232254] Lustre: server umount lustre-OST0001 complete [12845.728935] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [12856.038861] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12856.468496] LustreError: 340396:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12856.522809] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12861.742175] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [12867.050958] LustreError: 340397:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12867.072070] LustreError: 340397:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12871.208414] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12871.898126] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12876.340445] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [12880.287177] Lustre: 341536:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [12887.708031] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12888.079685] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12894.672156] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [12898.285057] LustreError: 341888:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12903.058740] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12904.349672] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [12904.352503] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [12911.630348] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [12921.091749] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12925.363226] Lustre: 343410:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [12930.024899] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12930.030749] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12930.053572] Lustre: Skipped 2 previous similar messages [12930.063797] Lustre: Skipped 1 previous similar message [12935.139046] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12935.158576] Lustre: Skipped 5 previous similar messages [12936.272795] Lustre: server umount lustre-MDT0000 complete [12940.262660] LustreError: 341412:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12940.288823] LustreError: 341412:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [12941.829936] LustreError: 342719:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785322492 with bad export cookie 1987978934765891485 [12941.832953] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12941.838930] LustreError: 342719:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12942.260679] Lustre: server umount lustre-MDT0001 complete [12957.310435] Lustre: server umount lustre-OST0000 complete [12960.747420] Lustre: 334394:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785322495/real 1785322495] req@ffff9f71608e1500 x1872046064780288/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785322511 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [12960.794911] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12960.819095] Lustre: Skipped 1 previous similar message [12961.762779] Lustre: server umount lustre-OST0001 complete [12966.389686] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [12968.081561] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [12970.211452] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 06:55:18 (1785322518) [12994.229282] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [13003.182332] loop0: detected capacity change from 0 to 1099511627776 [13004.046689] SGI XFS with ACLs, security attributes, quota, no debug enabled [13004.065478] *** VALIDATE xfs *** [13004.090496] XFS (loop0): Mounting V5 Filesystem [13004.115600] XFS (loop0): Ending clean mount [13005.734548] loop1: detected capacity change from 0 to 18691697672192 [13117.565495] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_hostid [13128.062668] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [13162.620084] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13162.987066] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13163.024578] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13163.132938] Lustre: lustre-MDT0000: new disk, initializing [13163.267658] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13163.270456] Lustre: Skipped 1 previous similar message [13163.297170] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13167.894863] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [13177.118046] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13177.249688] Lustre: 348546:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13177.336380] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13177.339434] Lustre: Skipped 1 previous similar message [13177.420163] Lustre: lustre-MDT0001: new disk, initializing [13177.585066] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13177.592529] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13182.264624] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [13190.408506] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13195.365234] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13201.431952] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13201.717117] Lustre: lustre-OST0000: new disk, initializing [13201.725682] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13201.734679] Lustre: 350513:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13203.530859] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13203.539215] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13203.665439] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13208.039288] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [13216.919216] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13220.774476] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [13220.946281] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [13225.127290] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [13225.383779] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [13229.545963] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [13229.549511] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13229.550444] LustreError: Skipped 1 previous similar message [13229.551404] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13229.567809] Lustre: Skipped 1 previous similar message [13239.782109] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13239.803679] Lustre: Skipped 4 previous similar messages [13241.312168] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [13241.546991] Lustre: server umount lustre-OST0000 complete [13249.510457] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13249.526421] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13260.256113] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [13260.257414] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13260.277982] Lustre: Skipped 6 previous similar messages [13260.751356] Lustre: server umount lustre-MDT0000 complete [13264.137042] LustreError: 348538:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785322814 with bad export cookie 1987978934765893039 [13264.138431] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13264.143554] LustreError: 348538:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [13264.560415] Lustre: server umount lustre-MDT0001 complete [13273.049480] XFS (loop0): Unmounting Filesystem [13277.180800] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 07:00:25 (1785322825) [13284.325688] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13284.991512] LustreError: 353073:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13285.077297] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13285.088918] Lustre: Skipped 2 previous similar messages [13289.148539] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [13296.457816] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13301.059518] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [13308.472423] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13314.000405] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13320.525741] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13326.860428] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [13334.158774] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13342.691381] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [13342.701388] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13342.718730] Lustre: Skipped 1 previous similar message [13342.723200] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13345.769413] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [13348.033439] Lustre: server umount lustre-OST0000 complete [13353.454168] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13357.613590] Lustre: server umount lustre-MDT0000 complete [13361.021414] LustreError: 353055:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785322911 with bad export cookie 1987978934765893921 [13361.025535] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13361.029146] LustreError: 353055:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [13361.294327] Lustre: server umount lustre-MDT0001 complete [13373.505205] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing unload_modules_local [13376.565588] Key type lgssc unregistered [13376.861860] LNet: 356714:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13376.876317] LNetError: 356714:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13376.888866] LNet: Removed LNI 192.168.201.114@tcp [13377.758846] Key type .llcrypt unregistered [13377.762952] Key type ._llcrypt unregistered [13403.158509] Key type ._llcrypt registered [13403.161084] Key type .llcrypt registered [13403.243536] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [13404.445825] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [13404.482240] alg: No test for adler32 (adler32-zlib) [13405.537808] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [13405.744926] LNet: Added LNI 192.168.201.114@tcp [8/256/0/180] [13407.400393] Key type lgssc registered [13408.374534] Lustre: Echo OBD driver; http://www.lustre.org/ [13414.023275] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [13414.037425] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13415.332297] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13419.314630] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [13424.614983] LustreError: 358272:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13424.636980] LustreError: 358272:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [13428.061615] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13428.317840] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13432.607376] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [13440.596435] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13446.668586] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13453.697599] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13454.070804] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13460.543980] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [13467.468884] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13477.857875] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13477.884757] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13479.524569] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.14@tcp (stopping) [13479.527695] Lustre: Skipped 1 previous similar message [13482.981807] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13482.996914] Lustre: Skipped 1 previous similar message [13483.787361] Lustre: server umount lustre-MDT0001 complete [13484.627989] LustreError: 358268:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.14@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13488.106702] LustreError: 358272:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13488.133448] LustreError: 358272:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [13493.221116] LustreError: 359513:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13493.239220] LustreError: 359513:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [13493.860416] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13493.970349] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13493.973599] Lustre: Found index 1 for lustre-MDT0001, updating log [13493.996499] Lustre: 358262:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13494.244285] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13498.379239] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [13499.391767] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13499.903084] LustreError: 361798:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13499.913645] LustreError: 361798:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13499.925916] LustreError: 358264:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13499.993734] LustreError: 361804:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.201.114@tcp: cfg command failed: rc = -17 [13499.999509] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [13499.999509] [13509.603893] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13509.606029] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13509.625450] Lustre: Skipped 1 previous similar message [13509.630037] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13509.642304] Lustre: Skipped 1 previous similar message [13514.728624] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13514.738491] Lustre: Skipped 2 previous similar messages [13521.376151] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [13521.570921] Lustre: server umount lustre-MDT0001 complete [13524.968616] LustreError: 358272:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13524.990986] LustreError: 358272:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [13529.686842] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13529.795230] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13529.797747] Lustre: Found index 1 for lustre-MDT0001, updating log [13529.834855] Lustre: 358260:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13530.044817] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13533.837218] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [13535.228480] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13535.737529] LustreError: 362915:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13535.741606] LustreError: 362915:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13535.746107] LustreError: 358264:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13535.749851] LustreError: 358264:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13535.843681] LustreError: 362919:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [13535.852800] LustreError: 362919:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [13535.860991] LustreError: 362919:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.201.114@tcp: cfg command failed: rc = -17 [13535.872806] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.201.114@tcp [13535.872806] [13839.340850] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13839.348912] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13839.352019] Lustre: Skipped 2 previous similar messages [13839.362072] Lustre: Skipped 5 previous similar messages [13845.289332] Lustre: server umount lustre-MDT0001 complete [13848.140461] LustreError: 358267:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.14@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13848.164232] LustreError: 358267:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [13853.819389] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13853.914579] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13853.921771] Lustre: Found index 1 for lustre-MDT0001, updating log [13853.967151] Lustre: 358261:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13854.322078] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13857.898581] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [13859.818882] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13860.859657] LustreError: 372420:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13860.865967] LustreError: 372420:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13860.876355] LustreError: 358264:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13860.886240] LustreError: 358264:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13860.943388] LustreError: 372423:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [13860.965449] LustreError: 372423:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.201.114@tcp: cfg command failed: rc = -114 [13860.977984] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [13860.977984] [14176.738118] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [14176.742521] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14176.751080] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14176.753194] Lustre: Skipped 3 previous similar messages [14182.768209] Lustre: server umount lustre-MDT0000 complete [14185.656953] LustreError: 358251:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785323736 with bad export cookie 4524371913270531568 [14185.658241] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14185.667343] LustreError: 358251:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [14187.488976] LustreError: 360765:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14187.494175] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14187.504594] LustreError: 360765:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [14187.521429] Lustre: Skipped 2 previous similar messages [14191.921720] Lustre: server umount lustre-MDT0001 complete [14194.617443] Lustre: server umount lustre-OST0000 complete [14201.225589] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_hostid [14207.661477] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [14231.552526] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 07:16:20 (1785323780) [14236.315541] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14236.484155] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [14236.503471] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [14236.571245] Lustre: lustre-MDT0000: new disk, initializing [14236.615113] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [14236.627616] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [14240.218690] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [14248.528683] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14248.676506] Lustre: 384147:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [14248.753561] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [14248.756381] Lustre: Skipped 1 previous similar message [14248.826714] Lustre: lustre-MDT0001: new disk, initializing [14248.882646] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [14248.912372] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [14248.921386] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [14252.368833] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [14260.165948] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [14264.864553] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [14269.953770] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14270.168900] Lustre: lustre-OST0000: new disk, initializing [14270.173433] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [14270.179323] Lustre: 386114:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [14270.270381] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [14274.813141] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_default_debug -1 all [14278.677831] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [14278.687210] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [14278.757217] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [14281.754208] Lustre: DEBUG MARKER: oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [14288.867783] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14288.871097] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14288.881920] Lustre: Skipped 1 previous similar message [14288.892304] Lustre: Skipped 10 previous similar messages [14293.468485] Lustre: server umount lustre-MDT0000 complete [14293.987708] LustreError: 385037:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14294.001938] LustreError: 385037:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [14295.854095] LustreError: 384142:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785323846 with bad export cookie 4524371913270533521 [14295.855179] LustreError: MGC192.168.201.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14295.858307] LustreError: 384955:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [14295.858453] LustreError: 384955:0:(client.c:1381:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@0000000058837de9 x1872046804208512/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 2 fl Rpc:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [14295.861187] LustreError: 384142:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [14296.057631] Lustre: server umount lustre-MDT0001 complete [14309.268305] Lustre: server umount lustre-OST0000 complete [14329.695297] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing set_hostid [14337.285205] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing load_modules_local [14359.729273] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 14036 sec ========= 07:18:28 (1785323908) [14360.820221] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 07:18:29 (1785323909) === [14363.448796] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 07:18:31 (1785323911) === [14391.979326] Lustre: DEBUG MARKER: oleg114-server.virtnet: executing unload_modules_local [14394.202715] Key type lgssc unregistered [14394.416564] LNet: 392475:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [14394.419920] LNetError: 392475:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [14394.441893] LNet: Removed LNI 192.168.201.114@tcp [14395.031822] Key type .llcrypt unregistered [14395.036426] Key type ._llcrypt unregistered