[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 453463791 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2608MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001010] APIC: Switch to symmetric I/O mode setup [ 0.003169] x2apic enabled [ 0.004004] Switched APIC routing to physical x2apic. [ 0.005008] kvm-guest: setup PV IPIs [ 0.008798] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009017] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010005] pid_max: default: 32768 minimum: 301 [ 0.011152] LSM: Security Framework initializing [ 0.012096] Yama: becoming mindful. [ 0.014040] SELinux: Initializing. [ 0.016029] *** VALIDATE selinux *** [ 0.023083] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027543] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028186] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029160] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030149] *** VALIDATE tmpfs *** [ 0.032397] *** VALIDATE proc *** [ 0.033176] *** VALIDATE cgroup *** [ 0.034005] *** VALIDATE cgroup2 *** [ 0.035163] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.036109] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.037005] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.038021] Spectre V2 : User space: Vulnerable [ 0.039005] Speculative Store Bypass: Vulnerable [ 0.041835] debug: unmapping init [mem 0xffffffff88459000-0xffffffff88460fff] [ 0.043143] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044546] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045017] ... version: 2 [ 0.046009] ... bit width: 48 [ 0.047005] ... generic registers: 4 [ 0.047922] ... value mask: 0000ffffffffffff [ 0.048011] ... max period: 00007fffffffffff [ 0.049009] ... fixed-purpose events: 3 [ 0.049894] ... event mask: 000000070000000f [ 0.050217] rcu: Hierarchical SRCU implementation. [ 0.052130] smp: Bringing up secondary CPUs ... [ 0.053377] x86: Booting SMP configuration: [ 0.054017] .... node #0, CPUs: #1 #2 #3 [ 0.057073] smp: Brought up 1 node, 4 CPUs [ 0.059007] smpboot: Max logical packages: 1 [ 0.060011] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.233018] node 0 deferred pages initialised in 172ms [ 0.236008] devtmpfs: initialized [ 0.237298] x86/mm: Memory block size: 128MB [ 0.239834] gcov: version magic: 0x41383552 [ 0.240644] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.244116] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.247381] pinctrl core: initialized pinctrl subsystem [ 0.248130] [ 0.248612] ************************************************************* [ 0.251012] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.253010] ** ** [ 0.256012] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.258009] ** ** [ 0.260009] ** This means that this kernel is built to expose internal ** [ 0.262008] ** IOMMU data structures, which may compromise security on ** [ 0.264008] ** your system. ** [ 0.266007] ** ** [ 0.268007] ** If you see this message and you are not debugging the ** [ 0.270007] ** kernel, report this immediately to your vendor! ** [ 0.272009] ** ** [ 0.274009] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.276008] ************************************************************* [ 0.278818] NET: Registered protocol family 16 [ 0.280440] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.283048] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.285048] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.289032] cpuidle: using governor menu [ 0.290556] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.294401] PCI: Using configuration type 1 for base access [ 0.297142] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.306114] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.307011] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.308127] cryptd: max_cpu_qlen set to 1000 [ 0.310236] ACPI: Added _OSI(Module Device) [ 0.311015] ACPI: Added _OSI(Processor Device) [ 0.312000] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.313010] ACPI: Added _OSI(Processor Aggregator Device) [ 0.318585] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.323627] ACPI: Interpreter enabled [ 0.325043] ACPI: PM: (supports S0 S3 S4 S5) [ 0.326010] ACPI: Using IOAPIC for interrupt routing [ 0.328078] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.331307] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.340919] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.343023] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.345009] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.348076] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.353441] acpiphp: Slot [2] registered [ 0.355081] acpiphp: Slot [5] registered [ 0.356084] acpiphp: Slot [6] registered [ 0.358078] acpiphp: Slot [7] registered [ 0.359174] acpiphp: Slot [8] registered [ 0.361078] acpiphp: Slot [9] registered [ 0.362105] acpiphp: Slot [10] registered [ 0.364105] acpiphp: Slot [3] registered [ 0.365079] acpiphp: Slot [4] registered [ 0.366076] acpiphp: Slot [11] registered [ 0.368090] acpiphp: Slot [12] registered [ 0.369073] acpiphp: Slot [13] registered [ 0.370056] acpiphp: Slot [14] registered [ 0.371056] acpiphp: Slot [15] registered [ 0.372062] acpiphp: Slot [16] registered [ 0.374084] acpiphp: Slot [17] registered [ 0.375081] acpiphp: Slot [18] registered [ 0.376055] acpiphp: Slot [19] registered [ 0.377069] acpiphp: Slot [20] registered [ 0.378076] acpiphp: Slot [21] registered [ 0.380086] acpiphp: Slot [22] registered [ 0.381072] acpiphp: Slot [23] registered [ 0.382134] acpiphp: Slot [24] registered [ 0.383016] acpiphp: Slot [25] registered [ 0.383806] acpiphp: Slot [26] registered [ 0.385051] acpiphp: Slot [27] registered [ 0.385784] acpiphp: Slot [28] registered [ 0.386043] acpiphp: Slot [29] registered [ 0.387095] acpiphp: Slot [30] registered [ 0.388057] acpiphp: Slot [31] registered [ 0.388949] PCI host bridge to bus 0000:00 [ 0.390012] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.391016] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.393010] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.394008] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.396013] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.398013] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.400175] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.402063] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.405454] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.414014] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.419722] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.423029] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.426021] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.429020] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.433040] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.435686] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.437033] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.439975] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.445018] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.460022] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.466023] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.473514] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.478017] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.483021] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.503016] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.511611] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.513997] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.519016] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.534017] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.542743] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.546015] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.552018] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.576020] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.585354] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.595000] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.601020] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.615013] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.629531] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.635015] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.649015] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.673019] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.687000] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.695014] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.705020] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.730017] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.748509] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.750466] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.754406] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.756374] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.759188] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.764152] iommu: Default domain type: Passthrough [ 0.765325] SCSI subsystem initialized [ 0.766000] ACPI: bus type USB registered [ 0.768119] usbcore: registered new interface driver usbfs [ 0.770070] usbcore: registered new interface driver hub [ 0.772113] usbcore: registered new device driver usb [ 0.774250] pps_core: LinuxPPS API ver. 1 registered [ 0.776011] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.779073] PTP clock support registered [ 0.783134] EDAC MC: Ver: 3.0.0 [ 0.785120] PCI: Using ACPI for IRQ routing [ 0.786000] NetLabel: Initializing [ 0.787007] NetLabel: domain hash size = 128 [ 0.789014] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.791102] NetLabel: unlabeled traffic allowed by default [ 0.794256] vgaarb: loaded [ 0.797021] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.800015] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.806216] clocksource: Switched to clocksource kvm-clock [ 0.918122] VFS: Disk quotas dquot_6.6.0 [ 0.919773] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.922270] *** VALIDATE ramfs *** [ 0.923161] *** VALIDATE hugetlbfs *** [ 0.924230] pnp: PnP ACPI init [ 0.926608] pnp: PnP ACPI: found 6 devices [ 0.940689] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.948737] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.951043] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.953403] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.956133] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.958884] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.962132] NET: Registered protocol family 2 [ 0.965022] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.970610] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.975262] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.980221] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.984624] TCP: Hash tables configured (established 65536 bind 65536) [ 0.987587] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.990793] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.994036] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.997597] NET: Registered protocol family 1 [ 1.000427] RPC: Registered named UNIX socket transport module. [ 1.002863] RPC: Registered udp transport module. [ 1.006718] RPC: Registered tcp transport module. [ 1.008415] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.010502] NET: Registered protocol family 44 [ 1.011990] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.013860] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.015692] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.017985] PCI: CLS 0 bytes, default 64 [ 1.019476] Unpacking initramfs... [ 2.596741] debug: unmapping init [mem 0xffff898d3cc54000-0xffff898d3ffbffff] [ 2.605537] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.608441] software IO TLB: mapped [mem 0x00000000b8c54000-0x00000000bcc54000] (64MB) [ 2.613263] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.182404] Initialise system trusted keyrings [ 3.184242] Key type blacklist registered [ 3.187329] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.197930] zbud: loaded [ 3.204483] *** VALIDATE nfs *** [ 3.206781] *** VALIDATE nfs4 *** [ 3.211447] pstore: using deflate compression [ 3.220892] Platform Keyring initialized [ 3.576799] NET: Registered protocol family 38 [ 3.579585] Key type asymmetric registered [ 3.583481] Asymmetric key parser 'x509' registered [ 3.585925] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.590118] io scheduler mq-deadline registered [ 3.597120] io scheduler kyber registered [ 3.602393] io scheduler bfq registered [ 3.607564] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.618269] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.621474] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.626175] ACPI: Power Button [PWRF] [ 3.632577] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.640231] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.664758] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.681309] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.719703] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.758614] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.812767] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.825443] Non-volatile memory driver v1.3 [ 3.828078] Linux agpgart interface v0.103 [ 3.883791] virtio_blk virtio1: [vda] 149768 512-byte logical blocks (76.7 MB/73.1 MiB) [ 3.889751] vda: detected capacity change from 0 to 76681216 [ 3.927214] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.937914] vdb: detected capacity change from 0 to 1073741824 [ 3.977666] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.983869] vdc: detected capacity change from 0 to 2621440000 [ 4.010963] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 4.016143] vdd: detected capacity change from 0 to 2621440000 [ 4.043858] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 4.051209] vde: detected capacity change from 0 to 4294967296 [ 4.080827] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 4.087269] vdf: detected capacity change from 0 to 4294967296 [ 4.103688] libphy: Fixed MDIO Bus: probed [ 4.126729] usbcore: registered new interface driver usbserial_generic [ 4.131426] usbserial: USB Serial support registered for generic [ 4.136171] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 4.145588] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 4.152267] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 4.154214] mousedev: PS/2 mouse device common for all mice [ 4.158742] rtc_cmos 00:05: RTC can wake from S4 [ 4.158746] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 4.172622] rtc_cmos 00:05: registered as rtc0 [ 4.180478] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 4.186358] intel_pstate: CPU model not supported [ 4.192211] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 4.203916] hid: raw HID events driver (C) Jiri Kosina [ 4.206160] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 4.206370] usbcore: registered new interface driver usbhid [ 4.219051] usbhid: USB HID core driver [ 4.224734] drop_monitor: Initializing network drop monitor service [ 4.228067] Initializing XFRM netlink socket [ 4.234922] NET: Registered protocol family 10 [ 4.240718] Segment Routing with IPv6 [ 4.243400] NET: Registered protocol family 17 [ 4.246855] mpls_gso: MPLS GSO support [ 4.260196] RAS: Correctable Errors collector initialized. [ 4.265170] AVX version of gcm_enc/dec engaged. [ 4.271768] AES CTR mode by8 optimization enabled [ 4.422573] sched_clock: Marking stable (4422552934, 0)->(5248814123, -826261189) [ 4.430968] registered taskstats version 1 [ 4.433179] Loading compiled-in X.509 certificates [ 4.436345] zswap: loaded using pool lzo/zbud [ 4.480841] Key type big_key registered [ 4.492362] Key type encrypted registered [ 4.493793] ima: No TPM chip found, activating TPM-bypass! [ 4.495887] ima: Allocated hash algorithm: sha1 [ 4.497886] ima: No architecture policies found [ 4.499844] evm: Initialising EVM extended attributes: [ 4.501809] evm: security.selinux [ 4.503285] evm: security.ima [ 4.504496] evm: security.capability [ 4.505878] evm: HMAC attrs: 0x1 [ 4.508645] rtc_cmos 00:05: setting system clock to 2026-09-06 23:36:41 UTC (1788737801) [ 4.516361] debug: unmapping init [mem 0xffffffff89403000-0xffffffff895fffff] [ 4.519544] debug: unmapping init [mem 0xffffffff88182000-0xffffffff88458fff] [ 4.529082] Write protecting the kernel read-only data: 28672k [ 4.534481] debug: unmapping init [mem 0xffffffff86803000-0xffffffff869fffff] [ 4.541722] debug: unmapping init [mem 0xffffffff87114000-0xffffffff871fffff] [ 4.644725] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 4.670416] systemd[1]: Detected virtualization kvm. [ 4.674284] systemd[1]: Detected architecture x86-64. [ 4.677588] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 4.740421] systemd[1]: No hostname configured. [ 4.744326] systemd[1]: Set hostname to . [ 4.748645] random: systemd: uninitialized urandom read (16 bytes read) [ 4.755328] systemd[1]: Initializing machine ID from random generator. [ 4.829669] random: ln: uninitialized urandom read (6 bytes read) [ 5.005230] random: systemd: uninitialized urandom read (16 bytes read) [ 5.009959] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 5.018914] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 5.028808] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. Starting Create Volatile Files and Directories... [ OK ] Listening on udev Control Socket. Starting Apply Kernel Variables... [ OK ] Listening on udev Kernel Socket. Starting Setup Virtual Console... [ OK ] Reached target Timers. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Slices. [ OK ] Started Memstrack Anylazing Service. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Paths. [ OK ] Reached target Sockets. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 6.387964] device-mapper: uevent: version 1.0.3 [ 6.393276] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 7.783763] random: fast init done [ 7.803066] virtio_net virtio0 ens2: renamed from eth0 [ 7.846559] scsi host0: ata_piix [ 7.914319] scsi host1: ata_piix [ 7.918268] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 7.923965] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 13.010506] random: crng init done [ 13.011649] random: 7 urandom warning(s) missed due to ratelimiting [ 13.610598] dracut-initqueue[592]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 14.650419] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 16.223733] printk: systemd: 26 output lines suppressed due to ratelimiting [ 16.685977] SELinux: Disabled at runtime. [ 16.778990] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 16.789609] systemd[1]: Detected virtualization kvm. [ 16.792158] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 17.664383] systemd[1]: initrd-switch-root.service: Succeeded. [ 17.668562] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 17.680062] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 17.687396] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 17.698237] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 17.710637] systemd[1]: Starting Journal Service... Starting Journal Service... [ 17.717371] systemd[1]: Listening on initctl Compatibility Named Pipe. [ OK ] Listening on initctl Compatibility Named Pipe. Activating swap /dev/disk/by-label/SWAP... [ OK ] Created slice system-sshd\x2dkeygen.slice. Starting Apply Kernel Variables... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ 17.791385] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Starting Remount Root and Kernel File Systems... [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on Process Core Dump Socket. [ OK ] Created slice system-getty.slice. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. Mounting POSIX Message Queue File System... Mounting Huge Pages File System... Mounting Kernel Debug File System... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Stopped target Initrd File Systems. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Reached target rpc_pipefs.target. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Journal Service. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Apply Kernel Variables. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 18.703757] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 19.595986] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 19.620826] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 20.164195] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 20.291452] EDAC sbridge: Ver: 1.1.2 [ 23.856574] Key type dns_resolver registered [ 24.399152] NFS: Registering the id_resolver key type [ 24.403965] Key type id_resolver registered [ 24.408056] Key type id_legacy registered [* ] A start job is running for Configur…-only root support (6s / no limit) [** ] A start job is running for Configur…-only root support (7s / no limit) [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Mark the need to relabel after reboot... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... Starting Login Service... [ OK ] Reached target sshd-keygen.target. [ OK ] Started irqbalance daemon. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... [ OK ] Started Login Service. Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Started OpenSSH server daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting System Logging Service... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Authorization Manager. [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg329-server login: [ 46.902991] libcfs: loading out-of-tree module taints kernel. [ 46.919147] Key type ._llcrypt registered [ 46.920083] Key type .llcrypt registered [ 46.960025] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_hostid [ 53.922739] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 54.465940] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 54.472203] alg: No test for adler32 (adler32-zlib) [ 55.415133] Lustre: Lustre: Build Version: 2.17.58_3_g671df26 [ 55.683746] LNet: Added LNI 192.168.203.129@tcp [8/256/0/180] [ 57.287174] Key type lgssc registered [ 57.788522] Lustre: Echo OBD driver; http://www.lustre.org/ [ 63.477326] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 75.749766] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 79.737079] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 79.745328] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 80.827632] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 80.840548] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 80.876650] Lustre: lustre-MDT0000: new disk, initializing [ 80.898609] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 80.904562] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 82.192353] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 86.920768] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 86.952925] Lustre: 6481:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 86.964290] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 86.966149] Lustre: Skipped 1 previous similar message [ 87.002981] Lustre: lustre-MDT0001: new disk, initializing [ 87.021879] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 87.028650] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 87.031812] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 88.284213] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 90.467347] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 93.507165] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 93.607562] Lustre: lustre-OST0000: new disk, initializing [ 93.610393] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 93.612845] Lustre: 8421:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 93.631200] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 95.651031] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 100.844521] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 100.848689] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 100.875841] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 101.188782] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 101.240774] Lustre: lustre-OST0001: new disk, initializing [ 101.242573] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 101.245218] Lustre: 9497:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 101.263801] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 103.233179] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 109.005327] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 109.035117] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 109.038388] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 109.062491] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 113.524804] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 115.269031] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing check_logdir /tmp/testlogs/ [ 116.645426] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing yml_node [ 117.916161] Lustre: DEBUG MARKER: Client: 2.17.58.3 [ 118.668359] Lustre: DEBUG MARKER: MDS: 2.17.58.3 [ 119.420544] Lustre: DEBUG MARKER: OSS: 2.17.58.3 [ 119.896243] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Sun Sep 6 19:38:37 EDT 2026 [ 125.019296] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 125.499976] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 138.207467] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 138.210745] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 138.214927] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 139.743769] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 139.744029] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 139.747626] Lustre: Skipped 2 previous similar messages [ 139.749877] Lustre: Skipped 2 previous similar messages [ 143.250240] Lustre: server umount lustre-MDT0000 complete [ 144.386942] LustreError: 6475:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788737941 with bad export cookie 1884458746096816889 [ 144.388104] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 144.390776] LustreError: 6475:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 144.863505] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 144.863796] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 144.867857] Lustre: Skipped 1 previous similar message [ 144.872647] Lustre: Skipped 1 previous similar message [ 149.983471] LustreError: 6488:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 149.983477] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 149.990153] LustreError: 6488:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 150.557185] Lustre: server umount lustre-MDT0001 complete [ 165.855086] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 165.886795] Lustre: server umount lustre-OST0000 complete [ 173.130319] Lustre: server umount lustre-OST0001 complete [ 175.133877] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_hostid [ 178.179942] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 189.641306] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 189.713887] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 189.725213] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 189.766543] Lustre: lustre-MDT0000: new disk, initializing [ 189.785461] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 189.790500] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 190.934249] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 194.676311] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 194.711924] Lustre: 15036:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 194.805925] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 195.976539] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 199.430807] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 201.093101] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 203.121695] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 203.221977] Lustre: lustre-OST0000: new disk, initializing [ 203.223372] Lustre: Skipped 1 previous similar message [ 203.224681] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 203.226967] Lustre: Skipped 2 previous similar messages [ 203.229366] Lustre: 17002:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 203.257164] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 203.259473] Lustre: Skipped 1 previous similar message [ 204.938639] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 204.988391] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 204.992455] Lustre: Skipped 1 previous similar message [ 204.994311] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 205.034449] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 208.315165] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 209.497615] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 209.559388] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 210.762424] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 210.836886] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 215.522314] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 215.522615] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 215.523310] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 215.523314] Lustre: Skipped 1 previous similar message [ 215.525293] LustreError: Skipped 1 previous similar message [ 215.529146] Lustre: Skipped 1 previous similar message [ 217.563766] Lustre: server umount lustre-OST0000 complete [ 220.640859] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 220.641125] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 220.647547] Lustre: Skipped 1 previous similar message [ 224.916237] Lustre: server umount lustre-MDT0000 complete [ 225.760141] LustreError: 15046:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 225.766670] LustreError: 15046:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 226.126209] LustreError: 15842:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788738023 with bad export cookie 1884458746096818408 [ 226.129163] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 226.131234] LustreError: 15842:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 226.238515] Lustre: server umount lustre-MDT0001 complete [ 227.784509] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 19:40:24 (1788738024) [ 230.053680] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 230.208273] LustreError: 19262:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 230.233809] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 231.640462] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 234.492812] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 235.955346] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 238.379654] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 240.239727] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 242.460070] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 244.440483] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 246.760710] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 258.862937] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 262.113547] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 262.114561] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 262.117950] Lustre: Skipped 2 previous similar messages [ 262.119776] Lustre: Skipped 4 previous similar messages [ 267.990611] Lustre: server umount lustre-MDT0000 complete [ 269.112120] LustreError: 19244:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788738066 with bad export cookie 1884458746096819290 [ 269.115232] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 269.116622] LustreError: 19244:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 269.213524] Lustre: server umount lustre-MDT0001 complete [ 280.672367] Lustre: server umount lustre-OST0000 complete [ 287.444525] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 290.829350] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 290.947280] LustreError: 23656:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 290.960342] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 290.963566] Lustre: Skipped 2 previous similar messages [ 292.210783] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 294.965773] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 296.412314] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 297.346381] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 299.506405] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 301.495526] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 304.431548] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 304.499359] Lustre: lustre-OST0001: new disk, initializing [ 304.501638] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 304.505046] Lustre: 25972:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 306.375807] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 306.379215] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 306.429401] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 306.521055] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 310.987593] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 313.532511] Lustre: Setting parameter general.obdfilter.*.client_cache_count=256 in log params [ 313.536345] Lustre: Skipped 1 previous similar message [ 316.899070] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 316.901110] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 316.904450] Lustre: Skipped 2 previous similar messages [ 316.907463] Lustre: Skipped 5 previous similar messages [ 322.517212] Lustre: server umount lustre-MDT0000 complete [ 323.732485] LustreError: 23638:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788738120 with bad export cookie 1884458746096820137 [ 323.736522] LustreError: 23638:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 323.736556] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 323.861848] Lustre: server umount lustre-MDT0001 complete [ 335.360857] Lustre: server umount lustre-OST0000 complete [ 346.841924] Lustre: server umount lustre-OST0001 complete [ 351.803604] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 354.779663] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 354.899176] LustreError: 28985:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 356.151852] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 358.762065] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 358.907452] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 358.910569] Lustre: Skipped 4 previous similar messages [ 360.306913] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 361.312198] Lustre: 30132:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 363.440883] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 364.579093] LustreError: 30485:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 365.458642] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 368.192703] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 369.962484] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 372.888786] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 373.956543] Lustre: 32008:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 379.359717] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 379.362109] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 379.366967] Lustre: Skipped 1 previous similar message [ 379.368581] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 379.370802] Lustre: Skipped 4 previous similar messages [ 381.460865] Lustre: server umount lustre-MDT0000 complete [ 382.636250] LustreError: 29735:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788738179 with bad export cookie 1884458746096821376 [ 382.639565] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 382.639851] LustreError: 29735:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 382.743497] Lustre: server umount lustre-MDT0001 complete [ 393.160152] Lustre: server umount lustre-OST0000 complete [ 404.566160] Lustre: server umount lustre-OST0001 complete [ 407.259048] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 19:43:24 (1788738204) [ 415.714152] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 418.834309] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 418.955959] LustreError: 34803:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 420.186504] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 422.779176] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 424.201070] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 425.108561] Lustre: 35951:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 427.153486] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 429.005548] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 431.692701] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 433.535830] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 436.515071] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 437.585361] Lustre: 37829:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 443.361910] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 443.363861] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 443.368371] Lustre: Skipped 6 previous similar messages [ 443.370155] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 443.372073] Lustre: Skipped 6 previous similar messages [ 444.820508] Lustre: server umount lustre-MDT0000 complete [ 445.951263] LustreError: 34786:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788738242 with bad export cookie 1884458746096822461 [ 445.955044] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 445.955589] LustreError: 34786:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 446.061845] Lustre: server umount lustre-MDT0001 complete [ 457.728546] Lustre: server umount lustre-OST0000 complete [ 468.114869] Lustre: server umount lustre-OST0001 complete [ 470.538830] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 19:44:27 (1788738267) [ 475.123852] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 478.190620] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 478.300888] LustreError: 39962:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 478.306799] LustreError: 39962:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 479.521127] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 482.064331] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 483.443037] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 484.328545] Lustre: 41110:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 486.302097] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 488.191184] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 490.728677] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 490.800298] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 490.802825] Lustre: Skipped 9 previous similar messages [ 492.545319] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 495.535224] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 496.950412] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 500.054126] Lustre: server umount lustre-MDT0000 complete [ 501.197220] LustreError: 40712:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788738298 with bad export cookie 1884458746096823546 [ 501.200598] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 501.202141] LustreError: 40712:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 507.369083] Lustre: server umount lustre-MDT0001 complete [ 509.983836] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 511.071106] LustreError: 43842:0:(import.c:339:ptlrpc_invalidate_import()) MGS: timeout waiting for callback (1 != 0) [ 511.073996] LustreError: 43842:0:(import.c:363:ptlrpc_invalidate_import()) @@@ still on sending list req@ffff898dbcafc380 x1875627582178944/t0(0) o250->MGC192.168.203.129@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 1788738308 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'ptlrpcd_00_01.0' uid:0 gid:0 projid:4294967295 [ 511.081536] LustreError: 43842:0:(import.c:373:ptlrpc_invalidate_import()) MGS: Unregistering RPCs found (0). Network is sluggish? Waiting for them to error out. [ 511.391832] LustreError: 3615:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff898dbcafe300 x1875627582180352/t0(0) o250->MGC192.168.203.129@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 512.782820] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 515.441219] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 516.874832] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 519.162857] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 531.029686] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 11 sec [ 531.942846] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 532.756519] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 533.258587] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 536.032619] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 540.629233] Lustre: server umount lustre-MDT0000 complete [ 541.812043] LustreError: 42987:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788738338 with bad export cookie 1884458746096824624 [ 541.815395] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 541.816918] LustreError: 42987:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 541.929494] Lustre: server umount lustre-MDT0001 complete [ 553.408291] Lustre: server umount lustre-OST0000 complete [ 564.818295] Lustre: server umount lustre-OST0001 complete [ 567.334710] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 19:46:04 (1788738364) [ 572.107647] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 575.401979] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 575.527802] LustreError: 47601:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 575.533308] LustreError: 47601:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 576.741305] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 579.209098] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 580.566896] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 581.467288] Lustre: 48750:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 581.470559] Lustre: 48750:0:(mgs_llog.c:1450:mgs_modify_param()) Skipped 1 previous similar message [ 583.442908] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 585.345752] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 588.019269] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 589.932890] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 592.877579] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 594.274910] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 599.522043] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 599.524789] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 599.526424] Lustre: Skipped 10 previous similar messages [ 599.528122] Lustre: Skipped 16 previous similar messages [ 603.539906] Lustre: server umount lustre-MDT0000 complete [ 604.794934] Lustre: server umount lustre-MDT0001 complete [ 615.947512] Lustre: server umount lustre-OST0000 complete [ 627.546682] Lustre: server umount lustre-OST0001 complete [ 630.682859] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 19:47:07 (1788738427) [ 631.478152] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 632.038537] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 19:47:09 (1788738429) [ 635.322623] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_hostid [ 638.062298] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 645.164897] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 645.253870] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 645.289810] Lustre: lustre-MDT0000: new disk, initializing [ 645.313562] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 646.614623] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 650.292708] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 652.532328] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 652.633555] Lustre: 54816:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 654.153605] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 654.162653] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 654.662848] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 658.444341] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 659.168378] Lustre: DEBUG MARKER: create test files [ 669.666384] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 675.462630] Lustre: server umount lustre-OST0000 complete [ 676.794337] Lustre: server umount lustre-MDT0000 complete [ 680.567114] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing unload_modules_local [ 681.605572] Key type lgssc unregistered [ 681.737764] LNet: 56425:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 681.740352] LNetError: 56425:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 681.748310] LNet: Removed LNI 192.168.203.129@tcp [ 682.096104] Key type .llcrypt unregistered [ 682.097310] Key type ._llcrypt unregistered [ 692.364881] Key type ._llcrypt registered [ 692.366008] Key type .llcrypt registered [ 692.403573] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 692.836106] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 692.844095] alg: No test for adler32 (adler32-zlib) [ 693.706238] Lustre: Lustre: Build Version: 2.17.58_3_g671df26 [ 693.790802] LNet: Added LNI 192.168.203.129@tcp [8/256/0/180] [ 695.375171] Key type lgssc registered [ 695.731226] Lustre: Echo OBD driver; http://www.lustre.org/ [ 698.301277] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 698.310803] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 699.447965] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 700.760219] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 703.164953] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 705.383181] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 705.508677] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 707.406750] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 709.605082] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 709.630445] Lustre: 59127:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 709.634405] Lustre: 59127:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 709.637792] Lustre: 59127:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 709.641353] Lustre: 59127:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 709.645829] Lustre: 59127:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 709.649789] Lustre: 59127:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 709.757386] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 712.841052] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 714.455820] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 719.841278] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 719.842140] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 719.850868] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 722.515568] Lustre: server umount lustre-OST0000 complete [ 723.944491] Lustre: server umount lustre-MDT0000 complete [ 727.496271] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing unload_modules_local [ 728.392924] Key type lgssc unregistered [ 728.520374] LNet: 60731:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 728.523891] LNetError: 60731:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 728.534259] LNet: Removed LNI 192.168.203.129@tcp [ 728.801482] Key type .llcrypt unregistered [ 728.802720] Key type ._llcrypt unregistered [ 739.486364] Key type ._llcrypt registered [ 739.488173] Key type .llcrypt registered [ 739.524816] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 739.890484] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 739.908751] alg: No test for adler32 (adler32-zlib) [ 740.761416] Lustre: Lustre: Build Version: 2.17.58_3_g671df26 [ 740.846515] LNet: Added LNI 192.168.203.129@tcp [8/256/0/180] [ 742.423120] Key type lgssc registered [ 742.771992] Lustre: Echo OBD driver; http://www.lustre.org/ [ 745.264573] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 745.271559] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 746.414185] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 747.729328] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 750.073135] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 752.319675] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 752.434553] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 754.330802] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 756.516242] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 756.535685] Lustre: 63533:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 756.540244] Lustre: 63533:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 756.543758] Lustre: 63533:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 756.546544] Lustre: 63533:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 756.550078] Lustre: 63533:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 756.552568] Lustre: 63533:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 756.701757] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 759.708532] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 761.825146] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 761.825440] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 761.833999] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 763.798549] Lustre: server umount lustre-OST0000 complete [ 767.160184] Lustre: server umount lustre-MDT0000 complete [ 770.758774] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing unload_modules_local [ 771.817716] Key type lgssc unregistered [ 771.946376] LNet: 65089:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 771.948699] LNetError: 65089:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 771.957305] LNet: Removed LNI 192.168.203.129@tcp [ 772.269969] Key type .llcrypt unregistered [ 772.271202] Key type ._llcrypt unregistered [ 778.359942] Key type ._llcrypt registered [ 778.361281] Key type .llcrypt registered [ 778.409742] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_hostid [ 784.727707] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 785.268419] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 785.275162] alg: No test for adler32 (adler32-zlib) [ 786.143234] Lustre: Lustre: Build Version: 2.17.58_3_g671df26 [ 786.235363] LNet: Added LNI 192.168.203.129@tcp [8/256/0/180] [ 787.815156] Key type lgssc registered [ 788.165608] Lustre: Echo OBD driver; http://www.lustre.org/ [ 799.707336] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 799.712824] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 800.805846] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 800.816990] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 800.853903] Lustre: lustre-MDT0000: new disk, initializing [ 800.878070] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 800.884805] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 802.272372] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 806.383511] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 806.432852] Lustre: 68208:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 806.458443] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 806.461487] Lustre: Skipped 1 previous similar message [ 806.499314] Lustre: lustre-MDT0001: new disk, initializing [ 806.531154] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 806.543065] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 806.547590] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 807.954292] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 811.668926] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 813.448872] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 815.624457] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 815.728843] Lustre: lustre-OST0000: new disk, initializing [ 815.730653] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 815.733377] Lustre: 70177:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 815.763658] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 817.620750] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 820.227178] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 820.232094] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 820.283503] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 821.394014] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 822.765764] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 822.834929] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 824.137460] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 824.205771] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 825.312400] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 825.313832] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 825.315847] LustreError: Skipped 1 previous similar message [ 825.322227] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 825.326215] Lustre: Skipped 1 previous similar message [ 826.898494] Lustre: server umount lustre-OST0000 complete [ 830.433689] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 830.434651] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 830.437445] Lustre: Skipped 2 previous similar messages [ 830.442464] Lustre: Skipped 1 previous similar message [ 834.267609] Lustre: server umount lustre-MDT0000 complete [ 835.511334] LustreError: 68201:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788738632 with bad export cookie 2083944766796578670 [ 835.514796] LustreError: 68201:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 835.515080] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 835.630859] Lustre: server umount lustre-MDT0001 complete [ 838.351775] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 19:50:35 (1788738635) [ 841.683919] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 853.875403] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_hostid [ 856.966459] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 868.084956] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 868.166853] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 868.177800] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 868.224236] Lustre: lustre-MDT0000: new disk, initializing [ 868.249296] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 868.256430] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 869.597599] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 873.747916] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 873.793036] Lustre: 76182:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 873.816897] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 873.818926] Lustre: Skipped 1 previous similar message [ 873.860974] Lustre: lustre-MDT0001: new disk, initializing [ 873.899856] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 873.914836] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 873.918539] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 875.321540] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 879.030135] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 880.887865] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 883.166064] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 883.270506] Lustre: lustre-OST0000: new disk, initializing [ 883.272198] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 883.274741] Lustre: 78148:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 883.302731] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 885.330086] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 887.803496] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 887.807877] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 887.836713] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 889.058678] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 890.530618] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 890.601266] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 891.974789] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 892.041347] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 892.897940] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 892.899536] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 892.900334] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 892.904276] Lustre: Skipped 1 previous similar message [ 892.910036] LustreError: Skipped 1 previous similar message [ 894.739839] Lustre: server umount lustre-OST0000 complete [ 898.016986] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 898.017561] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 898.020862] Lustre: Skipped 1 previous similar message [ 898.024012] Lustre: Skipped 2 previous similar messages [ 903.136120] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 903.137884] Lustre: Skipped 1 previous similar message [ 910.303136] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 910.376210] Lustre: server umount lustre-MDT0000 complete [ 911.604234] LustreError: 76176:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788738708 with bad export cookie 2083944766796579538 [ 911.607439] LustreError: 76176:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 911.607533] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 911.725035] Lustre: server umount lustre-MDT0001 complete [ 914.529122] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 19:51:51 (1788738711) [ 916.709206] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 916.893950] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 918.168649] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 921.012840] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 922.471772] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 924.819939] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 926.611284] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 928.823357] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 930.916219] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 933.352331] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 934.561597] Lustre: 83223:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting 80d51768-9460-4fed-86bb-ad11513c8c2b at adminstrative request [ 936.422556] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 936.427350] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 946.659203] Lustre: MGS: Client 80d51768-9460-4fed-86bb-ad11513c8c2b (at 0@lo) reconnecting [ 946.664320] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x1ceba8b96471c644 to 0x1ceba8b96471c8cf [ 946.668453] Lustre: MGC192.168.203.129@tcp: Connection restored to 0@lo (at 0@lo) [ 966.996042] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 967.066375] Lustre: lustre-OST0001: new disk, initializing [ 967.067994] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 967.070205] Lustre: 83621:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 967.103860] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 967.106018] Lustre: Skipped 2 previous similar messages [ 968.998935] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 972.616345] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 973.526316] Lustre: server umount lustre-OST0001 complete [ 975.340410] LustreError: 82376:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 975.341360] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 975.346867] LustreError: 82376:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 975.352672] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 975.356955] Lustre: Skipped 3 previous similar messages [ 980.450934] LustreError: 82374:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 980.455848] LustreError: 82374:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 980.811240] Lustre: server umount lustre-OST0000 complete [ 982.495913] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 985.570405] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 985.576427] Lustre: Skipped 2 previous similar messages [ 995.808131] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 995.810192] Lustre: Skipped 8 previous similar messages [ 996.319136] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 996.384751] Lustre: server umount lustre-MDT0000 complete [ 997.558205] LustreError: 80540:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788738794 with bad export cookie 2083944766796581071 [ 997.560306] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 997.565853] LustreError: 80540:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 997.679837] Lustre: server umount lustre-MDT0001 complete [ 1000.344579] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 19:53:17 (1788738797) [ 1000.892512] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 1001.680058] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 19:53:18 (1788738798) [ 1002.200360] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 1002.750343] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 19:53:19 (1788738799) [ 1003.259411] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 1003.826379] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 19:53:20 (1788738800) [ 1004.336485] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 1004.903142] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 19:53:21 (1788738801) [ 1005.436322] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 1007.496971] loop: module loaded [ 1007.501129] loop0: detected capacity change from 0 to 4294967296 [ 1007.510349] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 1008.049435] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 1009.163165] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 1010.461623] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 1019.018682] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1019.031373] LustreError: 87732:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 1019.033769] LustreError: 87732:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 1019.036059] LustreError: 87732:0:(tgt_mount.c:2585:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 1019.038780] LustreError: 87732:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 1025.826202] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_hostid [ 1029.108450] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 1040.724518] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1040.809290] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1040.818814] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1040.853453] Lustre: lustre-MDT0000: new disk, initializing [ 1040.873105] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1040.878778] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1042.211583] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 1046.322854] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1046.374357] Lustre: 90596:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1046.473668] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1047.737836] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 1051.651880] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1053.782879] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1056.146325] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1056.249198] Lustre: 92564:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1058.242234] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 1061.988951] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1063.478528] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1065.464246] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1065.497876] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1065.584806] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 2 sec [ 1066.874417] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1066.946703] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1070.562782] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1070.563036] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1070.564117] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1070.565299] LustreError: Skipped 1 previous similar message [ 1070.568762] Lustre: Skipped 1 previous similar message [ 1073.688516] Lustre: server umount lustre-OST0000 complete [ 1081.179620] Lustre: server umount lustre-MDT0000 complete [ 1082.514641] LustreError: 90589:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788738879 with bad export cookie 2083944766796581652 [ 1082.518340] LustreError: 90589:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1082.518374] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1082.662978] Lustre: server umount lustre-MDT0001 complete [ 1085.479916] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 19:54:42 (1788738882) [ 1087.626264] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1087.770325] LustreError: 94988:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1089.083436] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 1091.958994] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1093.359388] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 1096.331012] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1098.351245] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1100.918635] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1102.897334] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 1105.205238] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1106.587918] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1112.864989] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 6 sec [ 1114.689972] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1114.816395] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1117.423205] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1117.526945] Lustre: lustre-OST0001: new disk, initializing [ 1117.529813] Lustre: Skipped 2 previous similar messages [ 1117.532598] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1117.536375] Lustre: Skipped 3 previous similar messages [ 1117.539402] Lustre: 98398:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1119.841427] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 1124.316723] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1126.081481] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 1126.914708] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1126.917542] Lustre: Skipped 2 previous similar messages [ 1126.919490] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1126.945744] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1127.174951] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 1 sec [ 1128.475300] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 1128.547385] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1139.420964] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 1166.064084] Lustre: Failing over lustre-MDT0000 [ 1166.346946] Lustre: server umount lustre-MDT0000 complete [ 1167.861500] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1167.890209] LustreError: 94988:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1167.893923] Lustre: Skipped 3 previous similar messages [ 1167.952597] LustreError: 94988:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 1173.011731] LustreError: 94983:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1173.020831] LustreError: 94983:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 1181.243788] LustreError: 94985:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.203.29@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1181.275084] LustreError: 94985:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 1184.223156] Lustre: 66436:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788738964/real 1788738964] req@00000000a4925811 x1875628348603392/t0(0) o400->MGC192.168.203.129@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1788738980 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 1184.314991] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1197.647398] LustreError: 94984:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.203.29@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1197.680110] LustreError: 94984:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 23 previous similar messages [ 1202.526564] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 1202.528556] LDISKFS-fs (dm-0): recovery complete [ 1202.536504] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1209.828765] LustreError: 66435:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@000000004aaa9775 x1875628348615680/t0(0) o250->MGC192.168.203.129@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 1210.605484] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 1210.608613] Lustre: Skipped 6 previous similar messages [ 1210.743690] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 1213.052992] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 1213.054134] Lustre: 96337:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1214.301825] hrtimer: interrupt took 6171420 ns [ 1215.989384] Lustre: 94989:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1216.024388] Lustre: 94989:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 1216.131324] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1216.414364] LustreError: 101254:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1216.794397] LustreError: 101254:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1217.137723] Lustre: 101254:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1217.160646] Lustre: 101254:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 6 previous similar messages [ 1217.182352] LustreError: 101254:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1217.198242] LustreError: 101254:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 1217.747525] LustreError: 101254:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1217.781110] LustreError: 101254:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 1218.448110] LustreError: 101254:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1218.457138] LustreError: 101254:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 1218.803917] LustreError: 101254:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1218.823356] LustreError: 101254:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 1219.225391] Lustre: 101254:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1219.254470] Lustre: 101254:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 4 previous similar messages [ 1220.570583] LustreError: 101254:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1220.588690] LustreError: 101254:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 5 previous similar messages [ 1220.595367] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 1220.920303] LustreError: 101254:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1220.939631] LustreError: 101254:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 5 previous similar messages [ 1223.294411] Lustre: 101254:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1223.308833] Lustre: 101254:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 11 previous similar messages [ 1224.592022] LustreError: 101254:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1224.605653] LustreError: 101254:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 11 previous similar messages [ 1224.938042] LustreError: 101254:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1224.953883] LustreError: 101254:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 11 previous similar messages [ 1231.567916] Lustre: 101254:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1231.590229] Lustre: 101254:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 24 previous similar messages [ 1232.913760] LustreError: 101254:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1232.947907] LustreError: 101254:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 1233.287186] LustreError: 101254:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1233.309911] LustreError: 101254:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 1247.763698] Lustre: 101254:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1247.796180] Lustre: 101254:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 49 previous similar messages [ 1249.116596] LustreError: 101254:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1249.143396] LustreError: 101254:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 49 previous similar messages [ 1249.465258] LustreError: 101254:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1249.468670] LustreError: 101254:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 49 previous similar messages [ 1273.063382] Lustre: 101254:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 1273.074441] Lustre: 101254:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 1273.080477] Lustre: 101254:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client c16a18eb-b58a-42cf-9604-2996c95a0db2@192.168.203.29@tcp [ 1273.103113] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 1273.108577] Lustre: 101254:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@000000004b19f23a x1875628342182784/t0(8589934770) o35->c16a18eb-b58a-42cf-9604-2996c95a0db2@192.168.203.29@tcp:630/0 lens 392/0 e 0 to 0 dl 1788739080 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 1273.148691] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 1273.159389] Lustre: lustre-MDT0000: Denying connection for new client c16a18eb-b58a-42cf-9604-2996c95a0db2 (at 192.168.203.29@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 1273.177164] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 1273.214845] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1273.216961] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 1273.227271] Lustre: Skipped 2 previous similar messages [ 1273.254697] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:77 to 0x280000401:577) [ 1273.263405] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:545) [ 1286.625474] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1286.634059] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1286.644181] Lustre: Skipped 2 previous similar messages [ 1286.649741] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1286.656738] Lustre: Skipped 5 previous similar messages [ 1290.451695] Lustre: server umount lustre-OST0000 complete [ 1292.776600] LustreError: 96810:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1292.800182] LustreError: 96810:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 17 previous similar messages [ 1295.840484] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1295.849849] LustreError: Skipped 1 previous similar message [ 1308.136875] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1308.385411] Lustre: server umount lustre-OST0001 complete [ 1317.733683] Lustre: server umount lustre-MDT0000 complete [ 1320.340817] LustreError: 95014:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788739117 with bad export cookie 2083944766796632836 [ 1320.348788] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1320.352177] LustreError: 95014:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1320.579969] Lustre: server umount lustre-MDT0001 complete [ 1327.852953] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 19:58:44 (1788739124) [ 1334.317901] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1334.324886] Lustre: *** cfs_fail_loc=197, val=0*** [ 1349.920464] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 1355.039417] LustreError: 103230:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.129@tcp: failed processing log, type 4: rc = -110 [ 1360.176561] Lustre: *** cfs_fail_loc=197, val=0*** [ 1360.182065] Lustre: Skipped 65 previous similar messages [ 1386.253868] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 1392.167707] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1393.766877] Lustre: server umount lustre-OST0000 complete [ 1403.266746] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 19:59:58 (1788739198) [ 1428.923723] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1429.615207] LustreError: 104850:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1429.651235] LustreError: 104850:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 8 previous similar messages [ 1436.127445] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 1446.346757] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1450.213602] LustreError: 104881:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 1450.226400] LustreError: 104881:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 1450.234183] LustreError: 104881:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 21, retries 0, failed: rc = -5 [ 1450.790574] Lustre: server umount lustre-MDT0000 complete [ 1486.766837] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_hostid [ 1498.791310] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 1539.797863] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1539.911683] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1539.938396] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1540.000309] Lustre: lustre-MDT0000: new disk, initializing [ 1540.060631] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1540.066968] Lustre: Skipped 2 previous similar messages [ 1540.080573] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1542.579514] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 1553.661364] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1553.849131] Lustre: 109067:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1554.309157] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1557.149155] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 1565.418889] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1569.823309] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1574.561833] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1574.759204] Lustre: 111036:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1579.421369] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 1582.746675] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1586.407923] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1589.661854] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1589.866995] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1593.040915] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1593.261826] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1598.432785] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1598.441773] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1598.442313] LustreError: Skipped 1 previous similar message [ 1598.449473] Lustre: Skipped 5 previous similar messages [ 1598.450707] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1598.450712] Lustre: Skipped 12 previous similar messages [ 1609.183133] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1609.348866] Lustre: server umount lustre-OST0000 complete [ 1620.235963] Lustre: server umount lustre-MDT0000 complete [ 1624.033347] LustreError: 109077:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1624.075554] LustreError: 109077:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 1625.525583] LustreError: 109062:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788739422 with bad export cookie 2083944766796643434 [ 1625.527653] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1625.538512] LustreError: 109062:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1625.797221] Lustre: server umount lustre-MDT0001 complete [ 1636.349672] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 20:03:52 (1788739432) [ 1646.994841] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing unload_modules_local [ 1652.775399] Key type lgssc unregistered [ 1653.534048] LNet: 113527:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1653.543610] LNetError: 113527:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1653.575566] LNet: Removed LNI 192.168.203.129@tcp [ 1655.414914] Key type .llcrypt unregistered [ 1655.421173] Key type ._llcrypt unregistered [ 1679.170282] Key type ._llcrypt registered [ 1679.173238] Key type .llcrypt registered [ 1679.322858] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_hostid [ 1700.464461] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 1702.101216] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1702.116489] alg: No test for adler32 (adler32-zlib) [ 1703.739759] Lustre: Lustre: Build Version: 2.17.58_3_g671df26 [ 1704.353590] LNet: Added LNI 192.168.203.129@tcp [8/256/0/180] [ 1706.207259] Key type lgssc registered [ 1707.748402] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1762.935282] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1762.964301] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1764.370888] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1764.439780] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1764.555242] Lustre: lustre-MDT0000: new disk, initializing [ 1764.659174] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1764.688967] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1770.267506] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 1781.796885] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1781.989662] Lustre: lustre-OST0000: new disk, initializing [ 1781.992868] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1781.996437] Lustre: Skipped 1 previous similar message [ 1781.999575] Lustre: 117700:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1782.086361] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1788.418300] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 1789.600797] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1789.629074] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1789.669970] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1798.538405] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1798.787415] Lustre: lustre-OST0001: new disk, initializing [ 1798.793615] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1798.805524] Lustre: 118562:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1798.999511] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1804.363540] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 1804.367617] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 1804.418130] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 1806.760765] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 1819.633517] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1819.638754] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1819.659152] Lustre: Skipped 1 previous similar message [ 1819.675353] Lustre: Skipped 1 previous similar message [ 1822.289544] Lustre: server umount lustre-MDT0000 complete [ 1829.410123] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 1835.387081] LustreError: 116887:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788739632 with bad export cookie 11351930359953166844 [ 1835.415487] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1835.430939] LustreError: 116887:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1835.621619] Lustre: server umount lustre-OST0000 complete [ 1840.173928] Lustre: server umount lustre-OST0001 complete [ 1849.815836] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 20:07:25 (1788739645) [ 1873.962372] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 20:07:50 (1788739670) [ 1884.027542] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 1915.588104] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 20:08:30 (1788739710) [ 1968.652163] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_hostid [ 1988.857135] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 2037.786213] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2038.230275] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2038.267723] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2038.366246] Lustre: lustre-MDT0000: new disk, initializing [ 2038.429347] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2038.448212] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2043.936100] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 2054.982846] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2055.121499] Lustre: 126344:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2055.173546] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2055.177963] Lustre: Skipped 1 previous similar message [ 2055.271488] Lustre: lustre-MDT0001: new disk, initializing [ 2055.360073] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2055.422348] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2055.447019] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2059.506269] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 2068.099149] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2073.226643] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2080.001857] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2080.312415] Lustre: lustre-OST0000: new disk, initializing [ 2080.317261] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2080.331187] Lustre: 128313:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2080.429726] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2086.300459] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 2086.987746] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2086.994474] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2087.121467] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2095.430959] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2100.153578] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2100.368503] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2104.678619] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2104.895248] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2107.872730] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2107.875579] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2107.895212] LustreError: Skipped 1 previous similar message [ 2107.898898] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2107.908498] Lustre: Skipped 1 previous similar message [ 2107.918560] Lustre: Skipped 1 previous similar message [ 2112.891763] Lustre: server umount lustre-OST0000 complete [ 2117.088439] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2117.100961] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2117.114242] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2123.029536] Lustre: server umount lustre-MDT0000 complete [ 2126.837688] LustreError: 126338:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788739923 with bad export cookie 11351930359953168069 [ 2126.854619] LustreError: 126338:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2126.856823] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2127.416338] Lustre: server umount lustre-MDT0001 complete [ 2135.806585] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 20:12:11 (1788739931) [ 2142.613835] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2143.173835] LustreError: 130723:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2143.220229] LustreError: 130723:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2143.322191] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2147.572744] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 2148.326951] LustreError: 130724:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2153.449503] LustreError: 130723:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2155.707580] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2161.004739] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 2169.526910] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2174.776527] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2181.336939] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2181.608166] Lustre: lustre-OST0000: Not available for connect from 0@lo (not set up) [ 2181.615876] Lustre: Skipped 2 previous similar messages [ 2181.751734] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2181.769745] Lustre: Skipped 1 previous similar message [ 2189.257435] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 2196.598420] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2202.477401] Lustre: *** cfs_fail_loc=159, val=0*** [ 2219.108960] Lustre: lustre-MDT0000: Client 8409e9ed-0e48-42f0-88f7-5c42061cdb82 (at 192.168.203.29@tcp) reconnecting [ 2220.512201] Lustre: *** cfs_fail_loc=159, val=0*** [ 2220.514509] Lustre: Skipped 5 previous similar messages [ 2237.024785] Lustre: lustre-MDT0000: Client 8409e9ed-0e48-42f0-88f7-5c42061cdb82 (at 192.168.203.29@tcp) reconnecting [ 2243.552203] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2243.558040] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2243.573493] Lustre: Skipped 1 previous similar message [ 2243.578892] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2246.616465] Lustre: server umount lustre-OST0000 complete [ 2253.282837] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2253.296562] LustreError: Skipped 1 previous similar message [ 2253.300271] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2253.313954] Lustre: Skipped 1 previous similar message [ 2253.321040] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2253.329327] Lustre: Skipped 2 previous similar messages [ 2256.184452] Lustre: server umount lustre-MDT0000 complete [ 2258.942858] LustreError: 130703:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788740055 with bad export cookie 11351930359953168951 [ 2258.948426] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2258.950610] LustreError: 130703:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2259.281565] Lustre: server umount lustre-MDT0001 complete [ 2267.564884] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing unload_modules_local [ 2269.838426] Key type lgssc unregistered [ 2270.084618] LNet: 134494:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2270.096187] LNetError: 134494:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2271.142605] LNet: Removed LNI 192.168.203.129@tcp [ 2271.943437] Key type .llcrypt unregistered [ 2271.945728] Key type ._llcrypt unregistered [ 2291.162413] Key type ._llcrypt registered [ 2291.164634] Key type .llcrypt registered [ 2291.243147] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 2291.783807] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2291.825889] alg: No test for adler32 (adler32-zlib) [ 2292.777290] Lustre: Lustre: Build Version: 2.17.58_3_g671df26 [ 2292.929366] LNet: Added LNI 192.168.203.129@tcp [8/256/0/180] [ 2294.575196] Key type lgssc registered [ 2295.549225] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2301.526987] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2301.539994] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2302.844910] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2305.950799] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 2312.160818] LustreError: 135886:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2312.178742] LustreError: 135886:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 2312.191530] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2312.426296] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2315.575262] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 2320.516406] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2324.269339] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2328.645097] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2328.848524] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2333.557586] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 2338.444714] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2348.333766] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 2357.533296] Lustre: *** cfs_fail_loc=159, val=0*** [ 2373.797532] Lustre: lustre-MDT0000: Client 3951b3e9-ae0a-4dbe-b396-e06f7694aa50 (at 192.168.203.29@tcp) reconnecting [ 2377.251635] Lustre: *** cfs_fail_loc=159, val=0*** [ 2392.681509] Lustre: lustre-MDT0001: Client 3951b3e9-ae0a-4dbe-b396-e06f7694aa50 (at 192.168.203.29@tcp) reconnecting [ 2394.244429] Lustre: *** cfs_fail_loc=159, val=0*** [ 2394.273485] Lustre: Skipped 3 previous similar messages [ 2409.576317] Lustre: lustre-MDT0001: Client 3951b3e9-ae0a-4dbe-b396-e06f7694aa50 (at 192.168.203.29@tcp) reconnecting [ 2425.885608] Lustre: 138974:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2428.186521] Lustre: *** cfs_fail_loc=159, val=0*** [ 2428.189517] Lustre: Skipped 4 previous similar messages [ 2443.881619] Lustre: lustre-MDT0000: Client 735cebc6-8da8-4f7a-86d3-cc4d1e97a42f (at 192.168.203.29@tcp) reconnecting [ 2445.038604] Lustre: *** cfs_fail_loc=159, val=0*** [ 2445.045683] Lustre: Skipped 14 previous similar messages [ 2460.763717] Lustre: lustre-MDT0000: Client 735cebc6-8da8-4f7a-86d3-cc4d1e97a42f (at 192.168.203.29@tcp) reconnecting [ 2461.982841] Lustre: 139262:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2466.276687] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2466.281564] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2466.288982] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2466.295275] Lustre: Skipped 1 previous similar message [ 2466.304549] Lustre: Skipped 1 previous similar message [ 2467.448858] Lustre: server umount lustre-OST0000 complete [ 2471.395616] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2471.396080] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2471.405669] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2471.412522] Lustre: Skipped 1 previous similar message [ 2476.189793] Lustre: server umount lustre-MDT0000 complete [ 2476.515286] LustreError: 135887:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2476.531664] LustreError: 135887:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2478.584876] LustreError: 135867:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788740275 with bad export cookie 16210112570697800814 [ 2478.586827] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2478.595833] LustreError: 135867:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2478.821521] Lustre: server umount lustre-MDT0001 complete [ 2486.114651] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing unload_modules_local [ 2487.957548] Key type lgssc unregistered [ 2488.132912] LNet: 140309:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2488.138733] LNetError: 140309:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2488.152548] LNet: Removed LNI 192.168.203.129@tcp [ 2488.761571] Key type .llcrypt unregistered [ 2488.763454] Key type ._llcrypt unregistered [ 2511.258398] Key type ._llcrypt registered [ 2511.261475] Key type .llcrypt registered [ 2511.390886] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 2512.892641] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2513.115399] alg: No test for adler32 (adler32-zlib) [ 2514.347942] Lustre: Lustre: Build Version: 2.17.58_3_g671df26 [ 2514.858183] LNet: Added LNI 192.168.203.129@tcp [8/256/0/180] [ 2516.575248] Key type lgssc registered [ 2518.112783] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2525.729160] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2525.750560] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2527.261942] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2532.741071] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 2537.443610] LustreError: 141721:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2537.457416] LustreError: 141721:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 2540.627433] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2540.985516] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2545.060471] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 2552.441660] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2558.009446] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2565.387167] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2565.540364] LustreError: 143542:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2565.775306] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2571.277179] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 2572.357291] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 2572.964446] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 2580.425663] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2586.358257] Lustre: 144304:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2588.443542] Lustre: 144353:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2589.851688] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 2590.808911] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 2594.093548] Lustre: 144499:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 2594.989668] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 2595.849784] Lustre: 144600:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2601.952777] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2601.970226] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2602.001556] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2602.976109] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2602.983125] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2605.498120] Lustre: server umount lustre-OST0000 complete [ 2612.710942] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2612.719385] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2612.748310] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2612.756384] Lustre: Skipped 1 previous similar message [ 2615.256232] Lustre: server umount lustre-MDT0000 complete [ 2618.340122] LustreError: 141721:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2618.367233] LustreError: 141721:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2618.386675] LustreError: 143563:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788740415 with bad export cookie 13755424218880561865 [ 2618.390920] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2618.394986] LustreError: 143563:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2618.693175] Lustre: server umount lustre-MDT0001 complete [ 2629.650590] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing unload_modules_local [ 2633.098989] Key type lgssc unregistered [ 2633.650341] LNet: 145659:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2633.671826] LNetError: 145659:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2633.697766] LNet: Removed LNI 192.168.203.129@tcp [ 2634.798712] Key type .llcrypt unregistered [ 2634.802631] Key type ._llcrypt unregistered [ 2657.887185] Key type ._llcrypt registered [ 2657.889497] Key type .llcrypt registered [ 2657.975543] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 2658.956115] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2658.998243] alg: No test for adler32 (adler32-zlib) [ 2659.987252] Lustre: Lustre: Build Version: 2.17.58_3_g671df26 [ 2660.185118] LNet: Added LNI 192.168.203.129@tcp [8/256/0/180] [ 2661.823267] Key type lgssc registered [ 2662.809609] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2669.349429] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2669.374405] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2670.874753] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2675.075090] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 2680.292034] LustreError: 147067:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2680.320825] LustreError: 147067:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 2682.884062] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2683.253046] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2687.491164] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 2693.682786] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2698.577310] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2704.203980] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2704.564682] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2708.658532] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 2709.674847] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 2715.603577] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2715.821906] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 2720.460856] Lustre: *** cfs_fail_loc=159, val=0*** [ 2736.738355] Lustre: lustre-MDT0000: Client abc7ff1b-3cbd-4797-bf15-9443238eaff0 (at 192.168.203.29@tcp) reconnecting [ 2741.734022] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2741.738208] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2741.738604] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2741.745418] Lustre: Skipped 1 previous similar message [ 2743.671047] Lustre: server umount lustre-OST0000 complete [ 2746.849684] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2746.857273] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2746.864686] Lustre: Skipped 1 previous similar message [ 2746.876650] Lustre: Skipped 2 previous similar messages [ 2751.971482] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2751.974482] Lustre: Skipped 1 previous similar message [ 2754.938608] Lustre: server umount lustre-MDT0000 complete [ 2757.088344] LustreError: 147067:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2757.101338] LustreError: 147067:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2757.807330] LustreError: 147047:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788740554 with bad export cookie 3591394807739420839 [ 2757.812379] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2757.821356] LustreError: 147047:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2758.103317] Lustre: server umount lustre-MDT0001 complete [ 2766.885447] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing unload_modules_local [ 2769.148512] Key type lgssc unregistered [ 2769.390676] LNet: 150730:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2769.400493] LNetError: 150730:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2769.419624] LNet: Removed LNI 192.168.203.129@tcp [ 2770.129054] Key type .llcrypt unregistered [ 2770.133765] Key type ._llcrypt unregistered [ 2789.841809] Key type ._llcrypt registered [ 2789.844231] Key type .llcrypt registered [ 2789.914857] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 2790.550915] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2790.611276] alg: No test for adler32 (adler32-zlib) [ 2791.612281] Lustre: Lustre: Build Version: 2.17.58_3_g671df26 [ 2791.801842] LNet: Added LNI 192.168.203.129@tcp [8/256/0/180] [ 2793.431170] Key type lgssc registered [ 2794.330633] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2800.801457] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2800.821665] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2802.186161] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2805.526607] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 2811.734320] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2812.071622] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2815.576410] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 2821.128277] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2825.126760] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2829.898890] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2830.157072] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2834.703382] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 2836.329519] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 2840.008626] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2848.622581] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 2867.395752] Lustre: lustre-MDT0001: evicting 544c85e3-d0fe-4170-ad7d-ddc08842711c (at 192.168.203.29@tcp) by administrative request [ 2867.405535] Lustre: Skipped 1 previous similar message [ 2894.817416] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2894.821723] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2894.828945] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2894.838056] Lustre: Skipped 1 previous similar message [ 2894.846063] Lustre: Skipped 1 previous similar message [ 2899.937764] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2899.946152] Lustre: Skipped 1 previous similar message [ 2900.926863] Lustre: server umount lustre-OST0000 complete [ 2905.056377] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2905.062588] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2905.078776] Lustre: Skipped 1 previous similar message [ 2905.104430] Lustre: Skipped 1 previous similar message [ 2910.177444] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2910.183603] Lustre: Skipped 1 previous similar message [ 2910.627491] Lustre: server umount lustre-MDT0000 complete [ 2913.704595] LustreError: 152101:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788740710 with bad export cookie 2957420688266265803 [ 2913.713056] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2913.721381] LustreError: 152101:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2914.111758] Lustre: server umount lustre-MDT0001 complete [ 2925.038773] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing unload_modules_local [ 2928.307528] Key type lgssc unregistered [ 2928.602666] LNet: 156208:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2928.609457] LNetError: 156208:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2928.645549] LNet: Removed LNI 192.168.203.129@tcp [ 2929.516562] Key type .llcrypt unregistered [ 2929.521886] Key type ._llcrypt unregistered [ 2986.079119] Key type ._llcrypt registered [ 2986.081098] Key type .llcrypt registered [ 2986.140231] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 2986.816475] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2986.831514] alg: No test for adler32 (adler32-zlib) [ 2987.790960] Lustre: Lustre: Build Version: 2.17.58_3_g671df26 [ 2987.964142] LNet: Added LNI 192.168.203.129@tcp [8/256/0/180] [ 2989.631221] Key type lgssc registered [ 2990.445906] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2996.671321] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2996.701138] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2998.100654] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3002.717334] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 3008.494656] LustreError: 158671:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3008.508694] LustreError: 158671:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3010.804673] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3011.241466] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3015.065833] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 3021.256547] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3025.999781] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3032.201566] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3032.632454] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3036.726566] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 3038.166291] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 3043.896677] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 3045.058980] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3054.058395] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3054.061970] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3054.071244] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3054.078315] Lustre: Skipped 1 previous similar message [ 3059.174436] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3059.179595] Lustre: Skipped 2 previous similar messages [ 3060.074610] Lustre: server umount lustre-OST0000 complete [ 3064.290580] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3064.309492] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3064.320833] Lustre: Skipped 1 previous similar message [ 3069.298305] Lustre: server umount lustre-MDT0000 complete [ 3069.408497] LustreError: 158666:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3069.427238] LustreError: 158666:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3072.380200] LustreError: 158652:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788740869 with bad export cookie 15600989884905575009 [ 3072.387910] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3072.402084] LustreError: 158652:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3072.774259] Lustre: server umount lustre-MDT0001 complete [ 3081.032748] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing unload_modules_local [ 3083.365047] Key type lgssc unregistered [ 3083.645361] LNet: 162210:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3083.655460] LNetError: 162210:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3084.710859] LNet: Removed LNI 192.168.203.129@tcp [ 3085.513535] Key type .llcrypt unregistered [ 3085.517429] Key type ._llcrypt unregistered [ 3111.660185] Key type ._llcrypt registered [ 3111.662352] Key type .llcrypt registered [ 3111.896610] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 3112.951203] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3113.224953] alg: No test for adler32 (adler32-zlib) [ 3115.224325] Lustre: Lustre: Build Version: 2.17.58_3_g671df26 [ 3115.771919] LNet: Added LNI 192.168.203.129@tcp [8/256/0/180] [ 3117.647259] Key type lgssc registered [ 3119.585066] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3137.284474] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 20:28:53 (1788740933) [ 3153.667468] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_hostid [ 3164.105091] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 3210.325948] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3210.358747] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3211.542598] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3211.562305] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3211.604573] Lustre: lustre-MDT0000: new disk, initializing [ 3211.679603] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3211.695500] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3218.191301] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 3233.090034] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3233.158431] Lustre: 166472:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3233.207764] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3233.211186] Lustre: Skipped 1 previous similar message [ 3233.268741] Lustre: lustre-MDT0001: new disk, initializing [ 3233.332518] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3233.368359] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3233.385361] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3239.138661] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 3250.690984] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3257.451370] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3266.013807] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3266.443173] Lustre: lustre-OST0000: new disk, initializing [ 3266.449690] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3266.493805] Lustre: 168442:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3266.597053] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3273.197899] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 3274.832951] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3274.854732] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3274.969294] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3285.586745] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3300.832765] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3300.843937] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3300.859748] LustreError: Skipped 1 previous similar message [ 3300.897622] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3300.918217] Lustre: Skipped 1 previous similar message [ 3305.300878] Lustre: server umount lustre-OST0000 complete [ 3310.895532] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3310.908911] Lustre: Skipped 1 previous similar message [ 3310.916653] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3315.170242] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3315.176990] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3315.190955] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3317.310240] Lustre: server umount lustre-MDT0000 complete [ 3321.655059] LustreError: 166464:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788741118 with bad export cookie 5022329182610171480 [ 3321.662590] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3321.669210] LustreError: 166464:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3322.274258] Lustre: server umount lustre-MDT0001 complete [ 3334.087732] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing unload_modules_local [ 3337.391846] Key type lgssc unregistered [ 3337.778231] LNet: 170348:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3337.787107] LNetError: 170348:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3337.812491] LNet: Removed LNI 192.168.203.129@tcp [ 3338.933270] Key type .llcrypt unregistered [ 3338.938895] Key type ._llcrypt unregistered [ 3367.358687] Key type ._llcrypt registered [ 3367.361513] Key type .llcrypt registered [ 3367.499303] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 3368.887780] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3368.922173] alg: No test for adler32 (adler32-zlib) [ 3370.143396] Lustre: Lustre: Build Version: 2.17.58_3_g671df26 [ 3370.413904] LNet: Added LNI 192.168.203.129@tcp [8/256/0/180] [ 3372.167870] Key type lgssc registered [ 3373.682443] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3381.311711] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3381.347288] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3382.896349] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3387.286133] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 3392.489197] LustreError: 171765:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3392.509589] LustreError: 171765:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3395.290678] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3395.611994] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3399.983400] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 3407.019279] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3412.655679] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3419.308757] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3419.643931] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3425.932810] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 3427.863421] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 3432.882963] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3438.603387] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3438.631826] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3438.658832] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3438.673210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3438.694160] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3439.605578] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3439.624752] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3439.643504] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3439.662032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3439.690575] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3440.612956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3441.371471] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3441.378717] Lustre: Skipped 1 previous similar message [ 3441.393431] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3441.399517] Lustre: Skipped 1 previous similar message [ 3441.403066] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3441.407128] Lustre: Skipped 1 previous similar message [ 3441.409473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3441.434068] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3441.444143] Lustre: Skipped 1 previous similar message [ 3442.214248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3443.203090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3444.160258] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3444.172944] Lustre: Skipped 2 previous similar messages [ 3444.195493] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3444.204923] Lustre: Skipped 2 previous similar messages [ 3444.208937] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3444.213913] Lustre: Skipped 2 previous similar messages [ 3444.217761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3444.229526] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3444.244740] Lustre: Skipped 2 previous similar messages [ 3445.105905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3446.021661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3446.850877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3447.856430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3448.485542] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3448.497941] Lustre: Skipped 4 previous similar messages [ 3448.510348] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3448.519347] Lustre: Skipped 4 previous similar messages [ 3448.522504] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3448.528711] Lustre: Skipped 4 previous similar messages [ 3448.531761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3448.539732] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3448.544427] Lustre: Skipped 4 previous similar messages [ 3449.373464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3450.144401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3451.083478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3451.977653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3452.923908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3453.829402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3454.834657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3455.787109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3457.040389] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3457.049679] Lustre: Skipped 8 previous similar messages [ 3457.076245] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3457.092119] Lustre: Skipped 8 previous similar messages [ 3457.098146] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3457.108203] Lustre: Skipped 8 previous similar messages [ 3457.120355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3457.134501] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3457.146127] Lustre: Skipped 8 previous similar messages [ 3458.139725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3459.236135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3460.220138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3461.187383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3462.054992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3463.363664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3464.331468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3465.266952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3466.356797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3467.183466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3467.999629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3468.988108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3469.941216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3470.956435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3472.086706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3473.169523] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3473.181300] Lustre: Skipped 15 previous similar messages [ 3473.208372] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3473.220416] Lustre: Skipped 15 previous similar messages [ 3473.223638] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3473.233770] Lustre: Skipped 15 previous similar messages [ 3473.239735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3473.252261] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3473.265647] Lustre: Skipped 15 previous similar messages [ 3474.175608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3475.031964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3475.825237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3476.728488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3477.649596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3478.722155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3479.792231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3480.690937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3481.621411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3482.735141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3483.692961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3484.548319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3485.394466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3486.313129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3487.187429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3488.182779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3489.077376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3489.970709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3491.149484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3492.150852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3493.037858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3493.878984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3494.664731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3495.546757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3496.367915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3497.168169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3497.970136] LustreError: 171784:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3497.987053] LustreError: 171784:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3497.999070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3498.843728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3499.736854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3500.576446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3501.498482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3502.120970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3502.895443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3503.632425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3504.433539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3505.182198] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3505.186693] Lustre: Skipped 35 previous similar messages [ 3505.204628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3506.012593] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3506.025393] Lustre: Skipped 36 previous similar messages [ 3506.029976] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3506.036082] Lustre: Skipped 36 previous similar messages [ 3506.046505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3506.070346] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3506.078092] Lustre: Skipped 36 previous similar messages [ 3506.748928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3507.438861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3508.127370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3508.904118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3509.622437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3510.498177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3511.390911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3512.187288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3513.041475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3513.988973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3514.863715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3515.632800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3516.399138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3517.227490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3518.128050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3518.990895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3519.877420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3520.640330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3521.354126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3522.162239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3522.899524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3523.688323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3524.470258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3525.357559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3526.028438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3526.924243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3527.763364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3528.602886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3529.368394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3530.225895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3530.930030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3531.761906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3532.362514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3533.044505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3533.861933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3534.834775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3535.603483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3536.305127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3537.147619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3537.996195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3539.085139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3540.034184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3541.132859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3542.159996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3543.516098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3544.592899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3545.513718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3546.162662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3547.191533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3548.468508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3549.319943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3550.333276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3551.178611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3552.155415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3553.094289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3553.903877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3554.751929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3556.171369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3557.198740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3558.502898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3559.474374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3560.789443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3562.263695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3563.145055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3564.019983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3564.881505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3565.877373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3566.884473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3567.641347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3568.468886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3569.596788] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3569.613047] Lustre: Skipped 71 previous similar messages [ 3569.627394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3570.630819] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3570.651651] Lustre: Skipped 71 previous similar messages [ 3570.657730] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3570.665580] Lustre: Skipped 71 previous similar messages [ 3570.670568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3570.680943] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3570.686403] Lustre: Skipped 71 previous similar messages [ 3571.492242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3572.447457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3573.222169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3574.078912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3575.090718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3576.030759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3576.834310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3577.570333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3578.543741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3579.757430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3580.715844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3581.755072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3582.569714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3583.819789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3585.160367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3586.621745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3587.822485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3589.132606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3590.349137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3591.926257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3593.085674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3594.461660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3596.394607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3598.304558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3599.596909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3600.770873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3602.162752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3603.099568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3603.950371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3604.902397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3606.223180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3607.711417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3609.015467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3610.773495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3612.395051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3615.062271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3616.519172] LustreError: 171784:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3616.538510] LustreError: 171784:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3616.580277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3617.901657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3618.930965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3620.081555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3621.331892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3622.516664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3623.719704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3624.724667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3625.952542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3626.900844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3627.590538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3628.736907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3629.723457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3630.961030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3632.382219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3633.914018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3635.297218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3636.751065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3638.415633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3639.758335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3642.086942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3643.840243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3645.337444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3647.068941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3648.329300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3649.248517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3650.591827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3652.117808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3653.250435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3654.510928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3655.597777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3657.307845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3658.608928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3659.951655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3661.328888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3662.589536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3663.903821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3664.831206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3665.644108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3666.437748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3667.812879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3668.898915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3669.956909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3670.985192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3672.044190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3673.036411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3673.786851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3674.732609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3675.746967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3676.632834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3677.280077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3678.056448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3679.075202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3679.819575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3680.754067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3681.534713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3682.255905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3683.282510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3684.100436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3684.994680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3685.910958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3686.772831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3687.566720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3688.322524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3689.232361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3690.198346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3690.845221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3691.719291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3692.494271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3693.355920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3694.134392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3695.204724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3695.985817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3696.953338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3697.622879] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3697.630666] Lustre: Skipped 111 previous similar messages [ 3697.645594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3698.771126] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3698.785477] Lustre: Skipped 111 previous similar messages [ 3698.795037] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3698.809581] Lustre: Skipped 111 previous similar messages [ 3698.826872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3698.851932] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3698.874524] Lustre: Skipped 111 previous similar messages [ 3699.792249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3700.982982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3702.651909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3704.093905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3705.547424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3706.541324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3707.773941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3709.074792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3710.020717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3711.548395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3713.276671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3714.924991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3716.653732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3717.884282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3718.740555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3720.105800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3721.355416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3722.051699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3723.550424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3724.693318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3726.095518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3727.228345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3728.407774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3729.546059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3730.742676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3732.148939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3733.141255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3734.167690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3735.400065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3736.367499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3737.915813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3739.200492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3741.070729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3742.614111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3744.104751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3745.339074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3746.227953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3747.119642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3748.140609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3749.351295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3750.498798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3751.734459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3752.900663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3754.104095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3755.190043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3755.964522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3756.959949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3758.218062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3759.280655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3760.673532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3762.035873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3763.082554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3764.274148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3765.421491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3766.440335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3767.860340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3768.978784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3770.379815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3771.307357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3772.880860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3774.264442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3775.212811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3776.972948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3778.048429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3778.793566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3779.798414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3780.883325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3781.736735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3783.324741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3784.775054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3786.406968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3787.900137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3789.059639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3790.288553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3791.539200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3792.495115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3793.843523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3795.557147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3796.934840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3798.216161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3799.176167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3800.453430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3803.155060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3804.204734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3805.341552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3806.220129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3807.137905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3808.207862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3809.290968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3810.231476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3810.990471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3811.795081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3812.647774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3813.480095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3814.258202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3815.140304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3815.982107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3816.871772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3817.683116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3818.555231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3819.515840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3820.488512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3821.393640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3822.488343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3823.533685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3824.522588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3825.720943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3826.795299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3827.749357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3828.703797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3829.602762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3830.725096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3831.611863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3832.490678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3833.581114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3834.506056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3835.431990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3836.170325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3836.921546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3837.763424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3838.631345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3839.435313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3840.195119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3840.945814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3841.867764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3842.565922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3843.462129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3844.220274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3844.976949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3845.675159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3846.409264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3847.162969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3847.969574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3848.685776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3849.383788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3850.225985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3851.154386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3851.905075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3852.598795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3853.464869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3854.679575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3855.630718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3856.666067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3857.724795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3858.696716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3859.591619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3860.569273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3861.611710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3862.621275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3863.512783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3864.585988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3865.378483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3866.316159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3867.080559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3867.992287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3868.810893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3869.783354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3870.749266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3871.685790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3872.514352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3873.468849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3874.434531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3875.431877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3876.302434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3877.127603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3877.869786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3878.822982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3879.659927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3880.794669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3881.631130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3882.518669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3883.272918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3884.214987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3884.950276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3885.776550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3886.563209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3887.382511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3888.194457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3888.882497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3889.685979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3890.568375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3891.526664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3892.633607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3893.440446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3894.455101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3895.285654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3896.178691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3897.064618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3897.889353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3898.609269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3899.302803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3900.055198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3901.189694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3902.096169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3902.923731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3903.669797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3904.367171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3905.093657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3905.921024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3906.745449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3907.708245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3908.448876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3909.333786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3910.034627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3910.856389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3911.549973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3912.228186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3913.016367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3913.825212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3914.418604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3915.173570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3916.000772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3916.943807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3917.900509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3918.854720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3919.761848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3920.747600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3921.562149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3922.434819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3923.344866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3924.379481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3925.090867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3925.931326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3926.774675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3927.639889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3928.441256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3929.259303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3930.060443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3930.976484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3931.826496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3932.639163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3933.473825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3934.446524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3935.277517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3936.087440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3936.916876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3937.778987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3938.685846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3939.647609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3940.458822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3941.261531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3942.110191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3943.200689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3944.008579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3944.872040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3945.623488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3946.382711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3947.055889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3947.965290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3948.912491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3949.915515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3950.717650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3951.606626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3952.366369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3953.185422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3954.031358] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3954.042775] Lustre: Skipped 256 previous similar messages [ 3954.061111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3955.022546] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3955.037908] Lustre: Skipped 256 previous similar messages [ 3955.042801] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3955.050891] Lustre: Skipped 256 previous similar messages [ 3955.055977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3955.073996] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3955.082115] Lustre: Skipped 256 previous similar messages [ 3955.869036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3956.746207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3957.622296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3958.288325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3959.056199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3960.004838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3960.895102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3961.773884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3962.714040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3963.518413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3964.406575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3965.217538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3966.162475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3967.101481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3967.897657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3968.772663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3969.790436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3970.603063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3971.503961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3972.349975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3973.147968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3973.975249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3974.784226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3975.470823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3976.293229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3977.064673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3977.893463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3978.653700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3979.487354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3980.303518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3981.133076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3981.883383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3982.758543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3983.721968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3984.538791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3985.265315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3986.146056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3987.012713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3987.848550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3988.770839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3989.618690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3990.449160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3991.420315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3992.225473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3993.017282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3993.855851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3994.736643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3995.424885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3996.413595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3997.297920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3998.158809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3999.051823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4000.172933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4001.333138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4002.299501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4003.320525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4004.156445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4005.023430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4005.900407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4006.809510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4007.566703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4008.349781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4009.154349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4010.094427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4011.114258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4012.012184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4012.885305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4013.728104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4014.716421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4015.708270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4016.647717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4017.655517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4018.510990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4019.360953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4020.250616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4021.269892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4022.128764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4023.027367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4024.025872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4024.932146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4026.061740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4027.117820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4027.888789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4028.837602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4029.697444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4030.721574] LustreError: 171784:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4030.735223] LustreError: 171784:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4030.738218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4031.618391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4032.541176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4033.420604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4034.380151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4035.354337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4036.323553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4037.096303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4037.866245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4038.827987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4039.794838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4040.676545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4041.689057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4042.707206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4043.519765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4044.263816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4045.000789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4045.841920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4046.661641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4047.553595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4048.498246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4049.511077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4050.291805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4051.169591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4051.968543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4052.814900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4053.620497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4054.330988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4055.034390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4055.836731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4056.639278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4057.437029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4058.176926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4058.910947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4059.638716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4060.368460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4061.111170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4061.924111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4062.952507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4063.744957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4064.578991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4065.350199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4066.276202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4067.276839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4068.170821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4068.959788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4069.929352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4070.862532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4071.774778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4072.483545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4073.285505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4074.219900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4075.184385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4076.091564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4077.109465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4078.027331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4078.980256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4079.836217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4080.840545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4081.878951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4083.209311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4084.085120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4085.278138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4086.242451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4087.153677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4088.029897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4089.350399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4090.313667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4091.524667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4092.648683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4093.832234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4094.977870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4095.842118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4096.586642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4097.566464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4098.502361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4099.245557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4100.203579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4100.973687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4101.866360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4102.621464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4103.469201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4104.141379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4104.897536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4105.542992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4106.208895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4106.795628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4107.428778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4108.000075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4108.708338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4109.384759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4110.116467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4110.818430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4111.582653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4112.266809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4112.992645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4113.629323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4114.249933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4114.972894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4115.760841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4116.526201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4117.246233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4117.942382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4118.673742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4119.278471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4119.951976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4120.784757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4121.577588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4122.282847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4123.122424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4123.914578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4124.671503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4125.363921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4126.157368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4126.899530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4127.734875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4128.459937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4129.196900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4129.999057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4130.954961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4131.944247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4132.949538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4133.829369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4134.648983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4135.400911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4136.116534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4136.897741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4137.803321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4138.648875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4139.436964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4140.093131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4140.652266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4141.217256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4141.840264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4142.445818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4143.125945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4143.891302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4144.733337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4145.333817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4145.936764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4146.603888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4147.289579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4147.877233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4148.469126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4149.109988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4149.716179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4150.322194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4150.957742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4151.496102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4152.141355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4152.815767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4153.490615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4154.132736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4154.834745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4155.469752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4156.111799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4156.754156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4157.422833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4158.174247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4158.850467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4159.599196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4160.274421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4160.930276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4161.649532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4162.284212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4163.139843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4163.834587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4164.681394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4165.404712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4166.181778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4166.915820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4167.835028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4168.482700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4169.268581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4170.079814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4170.917231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4171.653297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4172.407231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4173.207592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4174.015947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4174.794129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4175.539751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4176.265338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4177.120875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4177.933995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4178.669418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4179.503746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4180.305274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4180.930477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4181.608919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4182.298191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4182.980874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4183.699321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4184.424816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4185.063280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4185.721852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4186.329148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4187.048456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4187.752745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4188.459949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4189.185627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4189.802836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4190.411958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4191.053203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4191.717554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4192.275432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4192.876724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4193.483120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4194.010044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4194.639676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4195.498477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4196.249267] LustreError: 171784:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4196.256713] LustreError: 171784:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4196.267941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4196.889487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4197.490317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4198.092968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4198.742486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4199.386353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4200.049590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4200.685048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4201.353502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4202.030327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4202.675435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4203.321443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4204.056252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4204.692614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4205.512583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4206.247474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4207.041524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4207.737340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4208.371593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4209.171561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4209.906203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4210.601653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4211.480878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4212.223205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4213.037412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4213.813946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4214.555245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4215.356563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4216.186213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4217.027858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4217.814613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4218.501061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4219.192165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4219.856954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4220.499972] LustreError: 171784:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4220.514314] LustreError: 171784:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4220.515095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4221.284511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4222.141523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4223.008839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4223.762724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4224.595591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4225.403440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4226.166649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4226.765167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4227.286831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4227.819845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4228.379335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4228.940901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4229.538830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4230.057723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4230.669395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4231.227535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4231.773253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4232.325807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4232.833842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4233.304760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4233.892458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4234.460163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4235.108065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4235.604056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4236.185486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4236.763138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4237.300247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4237.927817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4238.671553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4239.391413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4240.069575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4240.792075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4241.398886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4242.078767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4242.769632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4243.512608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4244.163876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4245.037524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4245.874482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4246.738956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4247.451345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4248.231732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4249.049510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4249.981802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4250.723447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4251.524687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4252.414050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4253.418850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4254.206268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4255.007531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4255.723911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4256.495294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4257.504166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4258.497982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4259.397432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4260.183405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4260.900552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4261.858946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4262.898217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4263.648747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4264.608148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4265.597498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4266.594368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4267.581192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4268.465870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4269.356219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4270.443301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4271.250635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4272.233050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4273.235162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4274.253841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4275.270037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4276.271896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4277.205923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4278.126767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4278.904875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4279.805436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4280.609217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4281.552317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4282.659469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4283.723928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4284.634693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4285.418624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4286.452215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4287.160580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4287.920729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4288.667650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4289.471815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4290.247959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4290.943497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4291.854602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4292.722761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4293.350408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4294.313861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4295.115347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4295.934222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4296.994840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4298.128287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4299.124217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4300.100963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4301.054952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4301.888150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4302.616469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4303.705893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4304.764748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4306.030659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4307.146247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4307.938538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4309.038920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4309.819822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4310.679647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4311.432860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4312.140761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4312.945519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4313.675027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4314.439550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4315.465382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4316.279737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4316.997700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4317.654463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4318.499832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4319.215272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4319.981699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4320.627092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4321.524927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4322.202694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4323.038533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4323.977780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4324.721406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4325.624841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4326.398895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4327.302166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4328.126545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4329.073770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4330.002528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4330.838366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4331.617126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4332.446812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4333.233406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4334.292493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4335.339532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4336.589578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4337.700730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4338.813602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4339.967484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4340.919678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4342.206153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4343.365121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4344.303408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4345.274183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4346.027766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4346.810416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4347.633675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4348.526965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4734.190518] LustreError: 173584:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 4734.203675] LustreError: 171784:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 5115.364077] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5115.371239] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5115.381971] Lustre: Skipped 485 previous similar messages [ 5115.385656] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5116.389900] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5116.402088] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5118.953890] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5118.979798] Lustre: Skipped 1 previous similar message [ 5121.736991] Lustre: server umount lustre-OST0000 complete [ 5126.631761] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5126.647456] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5132.673980] Lustre: server umount lustre-MDT0000 complete [ 5134.306557] LustreError: 173385:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5134.325410] LustreError: 173385:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 5135.889530] LustreError: 173607:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788742932 with bad export cookie 14652713477426213495 [ 5135.893617] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5135.896156] LustreError: 173607:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5136.206055] Lustre: server umount lustre-MDT0001 complete [ 5145.969117] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing unload_modules_local [ 5149.084780] Key type lgssc unregistered [ 5149.461835] LNet: 226989:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5149.468599] LNetError: 226989:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5149.497308] LNet: Removed LNI 192.168.203.129@tcp [ 5150.547227] Key type .llcrypt unregistered [ 5150.551853] Key type ._llcrypt unregistered [ 5175.134775] Key type ._llcrypt registered [ 5175.138534] Key type .llcrypt registered [ 5175.221530] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 5176.108920] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5176.132954] alg: No test for adler32 (adler32-zlib) [ 5177.063703] Lustre: Lustre: Build Version: 2.17.58_3_g671df26 [ 5177.234576] LNet: Added LNI 192.168.203.129@tcp [8/256/0/180] [ 5178.879177] Key type lgssc registered [ 5179.794674] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5186.351666] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5186.378713] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5187.915922] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5191.953658] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 5197.286274] LustreError: 228400:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5197.310655] LustreError: 228400:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5199.308897] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5199.702437] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5203.426959] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 5210.127502] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5215.253378] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5221.648776] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5222.045019] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5227.937607] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 5232.317050] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 5235.167475] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5240.391598] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5241.781935] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5241.792597] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5241.821682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.837365] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5241.838170] LustreError: 228420:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 5244.898908] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5244.906680] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5244.924652] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5247.979433] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5247.981934] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5248.018070] Lustre: Skipped 1 previous similar message [ 5249.935612] Lustre: server umount lustre-OST0000 complete [ 5261.895475] LustreError: 228420:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 5283.859546] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5284.219743] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5288.292441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.311093] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5291.929406] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 5298.550453] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.567361] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 5302.091957] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5304.613405] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 5311.992511] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff90e5c8157000.ost_server_uuid 50 [ 5314.330770] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff90e5c8157000.ost_server_uuid in IDLE state after 0 sec [ 5319.137248] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5319.143699] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5319.156667] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5319.183439] Lustre: Skipped 1 previous similar message [ 5319.217848] Lustre: Skipped 2 previous similar messages [ 5324.257588] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5324.940549] Lustre: server umount lustre-OST0000 complete [ 5332.962449] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5333.003096] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5333.065706] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5333.098676] Lustre: Skipped 1 previous similar message [ 5336.482292] Lustre: server umount lustre-MDT0000 complete [ 5339.617550] LustreError: 228395:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5339.654994] LustreError: 228395:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 5341.825368] LustreError: 231629:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788743138 with bad export cookie 750493851224320217 [ 5341.836430] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5341.846975] LustreError: 231629:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5342.841463] Lustre: server umount lustre-MDT0001 complete [ 5359.190988] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing unload_modules_local [ 5363.855445] Key type lgssc unregistered [ 5364.246103] LNet: 233610:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5364.288783] LNetError: 233610:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5364.320215] LNet: Removed LNI 192.168.203.129@tcp [ 5365.147600] Key type .llcrypt unregistered [ 5365.149853] Key type ._llcrypt unregistered [ 5396.970539] Key type ._llcrypt registered [ 5396.973034] Key type .llcrypt registered [ 5397.084623] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 5398.578656] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5398.603799] alg: No test for adler32 (adler32-zlib) [ 5399.695418] Lustre: Lustre: Build Version: 2.17.58_3_g671df26 [ 5399.964887] LNet: Added LNI 192.168.203.129@tcp [8/256/0/180] [ 5401.679269] Key type lgssc registered [ 5403.022815] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5415.971277] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5416.015793] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5417.510854] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5422.578346] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 5427.686293] LustreError: 235334:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5427.734080] LustreError: 235334:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5432.828032] LustreError: 235333:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5434.101509] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5434.417752] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5438.725692] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 5441.421966] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 5447.090813] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5447.336057] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5452.953732] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 5460.334191] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5460.488928] Lustre: lustre-OST0001: new disk, initializing [ 5460.491635] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5460.496450] Lustre: 237642:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5460.593687] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5462.645581] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 5467.174971] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 5469.858954] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5469.867756] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5469.983325] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5479.224142] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5484.953138] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 5500.911960] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5500.917422] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5500.923571] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5500.929680] Lustre: Skipped 3 previous similar messages [ 5500.945862] Lustre: Skipped 3 previous similar messages [ 5502.625892] Lustre: server umount lustre-MDT0000 complete [ 5505.202706] LustreError: 236068:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788743302 with bad export cookie 12277607825954599422 [ 5505.209936] LustreError: 236068:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5505.214130] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5505.480816] Lustre: server umount lustre-MDT0001 complete [ 5518.835545] Lustre: server umount lustre-OST0000 complete [ 5531.421674] Lustre: server umount lustre-OST0001 complete [ 5550.279205] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 5559.899523] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5560.342770] LustreError: 240751:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5560.438246] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5565.047455] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 5565.924487] LustreError: 240750:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5570.031118] LustreError: 240751:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5574.223848] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5574.673656] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5580.172737] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 5583.640893] Lustre: 241891:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5590.418772] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5591.029111] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5599.050217] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 5599.227463] LustreError: 242246:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5605.369254] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49760) [ 5606.738746] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5612.020618] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 5612.902988] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 5619.898563] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5623.409247] Lustre: 243765:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5640.679936] Lustre: mylustre-MDT0000-lwp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5640.690738] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5640.716254] Lustre: Skipped 2 previous similar messages [ 5640.743952] Lustre: Skipped 3 previous similar messages [ 5643.342539] Lustre: server umount mylustre-MDT0000 complete [ 5645.796236] LustreError: 240750:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5645.823813] LustreError: 240750:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 5646.852568] LustreError: 242269:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788743443 with bad export cookie 12277607825954602229 [ 5646.862562] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5646.867442] LustreError: 242269:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5647.299911] Lustre: server umount mylustre-MDT0001 complete [ 5661.641476] Lustre: server umount mylustre-OST0000 complete [ 5675.453687] Lustre: server umount mylustre-OST0001 complete [ 5696.302428] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 5704.007033] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5704.341187] LustreError: 246071:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5704.511738] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5704.525267] Lustre: Skipped 1 previous similar message [ 5708.835839] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 5717.706510] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5723.014866] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 5726.398197] Lustre: 247212:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5734.075451] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5741.362128] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 5748.588519] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5749.822268] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49792) [ 5749.833113] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 5754.574100] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 5762.534935] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5766.128240] Lustre: 249088:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5774.305240] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5774.314734] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5774.330373] Lustre: Skipped 1 previous similar message [ 5774.338584] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 5775.334069] LustreError: 248856:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5775.360451] LustreError: 248856:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 5775.396847] Lustre: server umount tfs-MDT0000 complete [ 5779.220284] LustreError: 246047:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788743576 with bad export cookie 12277607825954604119 [ 5779.224130] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5779.236475] LustreError: 246047:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 5779.632540] Lustre: server umount tfs-MDT0001 complete [ 5793.825787] Lustre: server umount tfs-OST0000 complete [ 5796.835799] Lustre: 234596:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788743577/real 1788743577] req@ffff898c94223b80 x1875633185691264/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788743593 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 5796.876231] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5796.883850] Lustre: Skipped 3 previous similar messages [ 5798.019716] Lustre: server umount tfs-OST0001 complete [ 5819.444845] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 5830.770021] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5831.601490] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5831.633949] Lustre: Skipped 3 previous similar messages [ 5837.879983] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 5846.494543] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5850.847882] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 5854.309056] Lustre: 252477:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5860.543118] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5866.416616] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 5874.430836] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5876.733235] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 5876.748019] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49824) [ 5879.597268] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 5886.059265] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5889.048618] Lustre: 254344:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5894.701584] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 21:14:50 (1788743690) [ 5912.552184] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5912.555416] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5912.561436] Lustre: Skipped 2 previous similar messages [ 5912.575759] Lustre: Skipped 3 previous similar messages [ 5917.669201] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5917.677255] Lustre: Skipped 2 previous similar messages [ 5924.319514] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5924.568945] Lustre: server umount lustre-MDT0000 complete [ 5927.427446] LustreError: 252854:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788743724 with bad export cookie 12277607825954605785 [ 5927.438176] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5927.904981] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5927.906356] LustreError: 251331:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5927.928153] Lustre: Skipped 1 previous similar message [ 5927.930233] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5927.947443] LustreError: 251331:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 5927.949672] Lustre: Skipped 6 previous similar messages [ 5933.807962] Lustre: server umount lustre-MDT0001 complete [ 5943.062822] Lustre: server umount lustre-OST0000 complete [ 5952.210492] Lustre: server umount lustre-OST0001 complete [ 5958.502062] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_hostid [ 5971.750572] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 6009.171267] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6009.426650] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6009.471252] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6009.638622] Lustre: lustre-MDT0000: new disk, initializing [ 6009.803778] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 6009.816300] Lustre: Skipped 3 previous similar messages [ 6009.850945] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6014.131808] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 6025.929158] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6026.187969] Lustre: 257527:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6026.241675] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6026.245171] Lustre: Skipped 1 previous similar message [ 6026.296450] Lustre: lustre-MDT0001: new disk, initializing [ 6026.512036] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6026.534957] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6031.352220] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 6041.095719] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6041.293180] Lustre: lustre-OST0000: new disk, initializing [ 6041.296348] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6041.301665] Lustre: 259111:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6043.335769] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6043.346244] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6043.526884] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6047.270310] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 6058.312382] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6058.534573] Lustre: lustre-OST0001: new disk, initializing [ 6058.541952] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 6058.559534] Lustre: 259981:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6059.976646] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 6059.992459] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 6060.151316] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 6065.814980] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 6077.127673] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 21:17:53 (1788743873) [ 6090.191396] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 21:18:06 (1788743886) [ 6105.064748] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6105.072292] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6105.083215] LustreError: Skipped 1 previous similar message [ 6105.088128] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6105.092369] Lustre: Skipped 2 previous similar messages [ 6105.109151] Lustre: Skipped 3 previous similar messages [ 6110.843378] Lustre: server umount lustre-OST0000 complete [ 6121.265204] Lustre: server umount lustre-MDT0000 complete [ 6125.813553] LustreError: 257520:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788743922 with bad export cookie 12277607825954607332 [ 6125.824087] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6125.831015] LustreError: 257520:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 6140.383488] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6140.682373] Lustre: server umount lustre-MDT0001 complete [ 6151.092677] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing unload_modules_local [ 6224.892775] Lustre: server umount lustre-OST0001 complete [ 6231.793662] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_hostid [ 6246.697840] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 6282.362883] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6282.557781] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6282.593563] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6282.697182] Lustre: lustre-MDT0000: new disk, initializing [ 6282.778670] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6282.787321] Lustre: Skipped 3 previous similar messages [ 6282.813534] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6286.622754] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 6295.154823] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6295.296758] Lustre: 264838:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6295.542502] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6299.035334] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 6306.516270] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6311.000945] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6316.763727] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6317.007706] Lustre: lustre-OST0000: new disk, initializing [ 6317.011905] Lustre: Skipped 1 previous similar message [ 6317.015930] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6317.020255] Lustre: Skipped 2 previous similar messages [ 6317.024019] Lustre: 266802:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6318.584850] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6318.596909] Lustre: Skipped 1 previous similar message [ 6318.604607] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6318.703406] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6322.270448] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 6330.091904] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6339.046412] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6339.047840] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6339.048749] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6339.048754] Lustre: Skipped 8 previous similar messages [ 6339.066665] Lustre: Skipped 5 previous similar messages [ 6341.542343] Lustre: server umount lustre-OST0000 complete [ 6346.720206] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6350.832479] Lustre: server umount lustre-MDT0000 complete [ 6354.138071] LustreError: 264831:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788744151 with bad export cookie 12277607825954608809 [ 6354.156196] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6354.170508] LustreError: 264831:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6354.803772] Lustre: server umount lustre-MDT0001 complete [ 6368.395342] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing unload_modules_local [ 6371.953712] Key type lgssc unregistered [ 6372.349613] LNet: 268562:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6372.368225] LNetError: 268562:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6372.392322] LNet: Removed LNI 192.168.203.129@tcp [ 6373.513211] Key type .llcrypt unregistered [ 6373.516790] Key type ._llcrypt unregistered [ 6399.518480] Key type ._llcrypt registered [ 6399.520950] Key type .llcrypt registered [ 6399.691895] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 6400.961188] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6401.026359] alg: No test for adler32 (adler32-zlib) [ 6402.177229] Lustre: Lustre: Build Version: 2.17.58_3_g671df26 [ 6402.626951] LNet: Added LNI 192.168.203.129@tcp [8/256/0/180] [ 6404.343175] Key type lgssc registered [ 6405.715978] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6413.395732] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6413.410351] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6414.867530] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6418.749405] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 6424.045583] LustreError: 270023:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6424.067394] LustreError: 270023:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 6426.861828] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6427.200220] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6432.835632] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 6440.195276] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6446.681782] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6452.664651] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6452.709750] LustreError: 271844:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6452.929631] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6458.212636] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 6459.383049] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 6466.372296] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6467.366103] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 6469.600306] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6469.610127] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6469.616252] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6469.651887] Lustre: Skipped 1 previous similar message [ 6474.726352] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6474.728770] Lustre: Skipped 1 previous similar message [ 6475.716452] Lustre: server umount lustre-OST0000 complete [ 6483.433066] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6483.451717] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6483.486899] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6483.506502] Lustre: Skipped 1 previous similar message [ 6484.966308] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6486.382026] Lustre: server umount lustre-MDT0000 complete [ 6490.084354] LustreError: 270022:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6490.114805] LustreError: 270022:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6490.244584] LustreError: 270003:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788744287 with bad export cookie 6643105939219925663 [ 6490.258647] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6490.263610] LustreError: 270003:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6490.804481] Lustre: server umount lustre-MDT0001 complete [ 6504.045699] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing unload_modules_local [ 6506.946688] Key type lgssc unregistered [ 6507.330523] LNet: 273606:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6507.347312] LNetError: 273606:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6507.359937] LNet: Removed LNI 192.168.203.129@tcp [ 6508.188316] Key type .llcrypt unregistered [ 6508.190586] Key type ._llcrypt unregistered [ 6524.582782] Key type ._llcrypt registered [ 6524.586484] Key type .llcrypt registered [ 6524.687697] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 6525.749982] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6525.813824] alg: No test for adler32 (adler32-zlib) [ 6527.056323] Lustre: Lustre: Build Version: 2.17.58_3_g671df26 [ 6527.331790] LNet: Added LNI 192.168.203.129@tcp [8/256/0/180] [ 6529.040777] Key type lgssc registered [ 6530.262450] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6540.173843] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6540.192788] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6555.743583] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 6560.863462] LustreError: 274797:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.129@tcp: failed processing log, type 4: rc = -110 [ 6587.487210] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6593.661977] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 6600.878059] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6602.788900] Lustre: server umount lustre-OST0000 complete [ 6618.948771] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing unload_modules_local [ 6621.891455] Key type lgssc unregistered [ 6622.216941] LNet: 276427:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6622.231234] LNetError: 276427:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6622.252753] LNet: Removed LNI 192.168.203.129@tcp [ 6623.146589] Key type .llcrypt unregistered [ 6623.153409] Key type ._llcrypt unregistered [ 6659.545378] Key type ._llcrypt registered [ 6659.551154] Key type .llcrypt registered [ 6659.635657] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 6660.629958] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6660.676271] alg: No test for adler32 (adler32-zlib) [ 6661.779364] Lustre: Lustre: Build Version: 2.17.58_3_g671df26 [ 6662.082670] LNet: Added LNI 192.168.203.129@tcp [8/256/0/180] [ 6663.759285] Key type lgssc registered [ 6664.738293] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6676.828496] loop0: detected capacity change from 0 to 204800000 [ 6677.059336] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6677.091241] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6677.539276] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6678.587756] loop0: detected capacity change from 0 to 204800000 [ 6678.803306] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6678.825436] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6679.318965] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6680.286381] loop0: detected capacity change from 0 to 204800000 [ 6680.486895] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6680.508355] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6680.634535] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6681.638758] loop0: detected capacity change from 0 to 204800000 [ 6681.881898] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6682.061851] print_req_error: 1 callbacks suppressed [ 6682.061862] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6683.140650] loop0: detected capacity change from 0 to 204800000 [ 6683.191244] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6687.740709] loop0: detected capacity change from 0 to 204800000 [ 6687.782953] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6691.784648] loop0: detected capacity change from 0 to 204800000 [ 6691.848263] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6695.856210] loop0: detected capacity change from 0 to 204800000 [ 6695.897960] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6700.113202] loop0: detected capacity change from 0 to 204800000 [ 6700.206293] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6700.268821] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6700.400265] Lustre: 279835:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6700.413661] Lustre: 279835:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 6700.434607] Lustre: 279835:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 6700.456794] Lustre: 279835:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 6700.464765] Lustre: 279835:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 6700.473794] Lustre: 279835:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 6700.491842] Lustre: 279835:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 6700.507954] Lustre: 279835:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 6700.525590] Lustre: 279835:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 6700.552974] Lustre: 279835:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 6700.564539] Lustre: 279835:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 6700.586853] Lustre: 279835:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 6700.599556] Lustre: 279835:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 6700.619176] Lustre: 279835:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 6706.459756] Lustre: server umount MGS complete [ 6707.482162] loop0: detected capacity change from 0 to 204800000 [ 6707.548131] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6708.080922] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6708.137226] Lustre: lustre-MDT0000: Aborting client recovery [ 6709.100078] loop1: detected capacity change from 0 to 204800000 [ 6709.306915] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6709.370057] Lustre: 280266:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6709.378948] Lustre: 280266:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 6709.387560] Lustre: 280266:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 6709.397250] Lustre: 280266:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 6709.410692] Lustre: 280266:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 6709.427516] Lustre: 280266:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 6709.448213] Lustre: 280266:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 6709.477489] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6709.715490] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6709.791576] Lustre: lustre-MDT0001: Aborting client recovery [ 6710.674221] loop2: detected capacity change from 0 to 204800000 [ 6710.782482] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6710.924294] Lustre: 280344:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6710.937705] Lustre: 280344:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 6710.949951] Lustre: 280344:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 6710.959172] Lustre: 280344:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 6710.969212] Lustre: 280344:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 6710.978498] Lustre: 280344:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 6710.991224] Lustre: 280344:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 6711.099457] Lustre: Found index 0 for lustre-OST0000, updating log [ 6711.132380] Lustre: 280363:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6711.187791] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6711.218365] Lustre: lustre-OST0000: Aborting recovery [ 6712.025483] loop3: detected capacity change from 0 to 204800000 [ 6712.170310] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6712.289322] Lustre: 280428:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6712.312191] Lustre: 280428:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 6712.333996] Lustre: 280428:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 6712.344923] Lustre: 280428:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 6712.357266] Lustre: 280428:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 6712.367408] Lustre: 280428:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 6712.385899] Lustre: 280428:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 6712.406445] Lustre: Found index 1 for lustre-OST0001, updating log [ 6712.432704] Lustre: 280433:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6712.543947] Lustre: lustre-OST0001: Aborting recovery [ 6717.973661] LustreError: 280210:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 9, retries 0, failed: rc = -108 [ 6718.952818] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 6718.953493] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 6718.999276] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 6719.033744] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 6719.973853] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 6719.978120] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 6721.324324] LustreError: 280177:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1028us: rc = -2 [ 6721.826361] LustreError: 280176:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1095us: rc = -2 [ 6721.854337] LustreError: 280176:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 19 previous similar messages [ 6735.328907] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6735.330513] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6735.346289] Lustre: Skipped 3 previous similar messages [ 6735.372318] Lustre: Skipped 3 previous similar messages [ 6740.448641] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6740.460911] Lustre: Skipped 3 previous similar messages [ 6740.986988] Lustre: server umount lustre-MDT0000 complete [ 6742.040453] LustreError: 280160:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788744539 with bad export cookie 13460813565399776319 [ 6742.054984] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6745.568515] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6745.569718] LustreError: 280176:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6745.570124] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6745.596882] Lustre: Skipped 1 previous similar message [ 6745.628398] LustreError: 280176:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 6747.617716] LustreError: 280440:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6747.620885] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6747.647819] Lustre: Skipped 2 previous similar messages [ 6748.484705] Lustre: server umount lustre-MDT0001 complete [ 6755.828244] Lustre: server umount lustre-OST0000 complete [ 6771.168229] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6771.592746] Lustre: server umount lustre-OST0001 complete [ 6779.704150] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 21:29:35 (1788744575) [ 6807.847250] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_hostid [ 6817.272627] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 6852.017198] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6852.354988] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6852.403948] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6852.457225] Lustre: lustre-MDT0000: new disk, initializing [ 6852.555872] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6852.563528] Lustre: Skipped 1 previous similar message [ 6852.581680] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6857.833245] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 6869.699078] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6869.909337] Lustre: 284504:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6870.026288] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6870.038483] Lustre: Skipped 1 previous similar message [ 6870.199587] Lustre: lustre-MDT0001: new disk, initializing [ 6870.358285] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6870.426082] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6870.448755] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6876.373580] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 6886.229924] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6891.299356] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6898.743872] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6899.238737] Lustre: lustre-OST0000: new disk, initializing [ 6899.246521] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6899.254913] Lustre: 286472:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6899.375681] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6904.914401] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6904.922336] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6905.042931] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6907.076924] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 6916.164604] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6921.444918] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 6932.240451] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 6935.885281] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 6987.233482] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6987.238859] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6987.248571] LustreError: Skipped 1 previous similar message [ 6987.251536] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6987.311202] Lustre: Skipped 1 previous similar message [ 6990.202640] Lustre: server umount lustre-OST0000 complete [ 6997.473694] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6997.482580] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6997.491521] Lustre: Skipped 1 previous similar message [ 6997.514430] Lustre: Skipped 2 previous similar messages [ 7001.043058] Lustre: server umount lustre-MDT0000 complete [ 7002.600134] LustreError: 288146:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7002.629394] LustreError: 288146:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7005.027423] LustreError: 287328:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788744802 with bad export cookie 13460813565399779028 [ 7005.038036] LustreError: 287328:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7005.054385] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7005.305600] Lustre: server umount lustre-MDT0001 complete [ 7014.088586] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7018.851382] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 7023.905638] Lustre: server umount MGS complete [ 7035.811672] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7036.279865] LustreError: 290293:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7036.311360] LustreError: 290306:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 7036.422887] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7040.953076] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 7041.525068] LustreError: 290294:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7049.667685] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7054.735731] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 7063.119988] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7069.145785] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7076.694802] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7077.176852] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7077.193923] Lustre: Skipped 1 previous similar message [ 7083.837408] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 7092.414408] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7126.497708] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7126.512377] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7126.523854] Lustre: Skipped 1 previous similar message [ 7126.546377] Lustre: Skipped 1 previous similar message [ 7131.927033] Lustre: server umount lustre-OST0000 complete [ 7136.738527] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7141.860359] LustreError: 291022:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7141.874522] LustreError: 291022:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7141.927661] Lustre: server umount lustre-MDT0000 complete [ 7145.436255] LustreError: 293018:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788744942 with bad export cookie 13460813565399780869 [ 7145.439430] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7145.453139] LustreError: 293018:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 7145.749654] Lustre: server umount lustre-MDT0001 complete [ 7155.985317] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing unload_modules_local [ 7158.591398] Key type lgssc unregistered [ 7158.893641] LNet: 294239:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7158.904114] LNetError: 294239:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7158.917560] LNet: Removed LNI 192.168.203.129@tcp [ 7159.703366] Key type .llcrypt unregistered [ 7159.706854] Key type ._llcrypt unregistered [ 7194.508706] Key type ._llcrypt registered [ 7194.510815] Key type .llcrypt registered [ 7194.613928] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_hostid [ 7213.857367] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 7215.889873] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7215.912753] alg: No test for adler32 (adler32-zlib) [ 7217.240517] Lustre: Lustre: Build Version: 2.17.58_3_g671df26 [ 7217.570641] LNet: Added LNI 192.168.203.129@tcp [8/256/0/180] [ 7219.295170] Key type lgssc registered [ 7220.232958] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7256.149651] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7256.175672] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7257.460922] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7257.496628] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7257.601699] Lustre: lustre-MDT0000: new disk, initializing [ 7257.764283] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7257.782169] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7262.132532] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 7272.287602] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7272.398976] Lustre: 298193:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7272.474502] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7272.478885] Lustre: Skipped 1 previous similar message [ 7272.542338] Lustre: lustre-MDT0001: new disk, initializing [ 7272.619155] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7272.650493] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7272.658088] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7277.570418] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 7287.874898] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7293.763534] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7301.121198] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7301.444727] Lustre: lustre-OST0000: new disk, initializing [ 7301.456462] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7301.466188] Lustre: 300161:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7301.564929] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7308.874608] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 7309.399548] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7309.413147] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7309.595775] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7318.901346] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7325.829767] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 7335.379815] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 7340.928085] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 7347.724736] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 7396.843918] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7396.846542] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7396.857751] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7396.875694] Lustre: Skipped 1 previous similar message [ 7398.911279] Lustre: server umount lustre-OST0000 complete [ 7406.062895] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7406.081200] LustreError: Skipped 1 previous similar message [ 7406.087510] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7406.109392] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7406.114450] Lustre: Skipped 1 previous similar message [ 7412.199976] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7412.208105] Lustre: Skipped 3 previous similar messages [ 7416.287328] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7416.289373] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7416.306455] Lustre: Skipped 1 previous similar message [ 7416.491877] Lustre: server umount lustre-MDT0000 complete [ 7419.864084] LustreError: 300160:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788745216 with bad export cookie 14044145969864514531 [ 7419.869486] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7419.887836] LustreError: 300160:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7420.205531] Lustre: server umount lustre-MDT0001 complete [ 7429.029539] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7433.582866] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 7438.148372] Lustre: server umount MGS complete [ 7448.543065] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7449.079919] LustreError: 303996:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 7449.137110] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7453.635297] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 7454.177041] LustreError: 303983:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7454.202064] LustreError: 303983:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7459.309974] LustreError: 303984:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7462.050340] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7462.364244] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7466.628630] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 7474.003295] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7479.444984] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7486.366430] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7486.701303] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7492.645932] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 7500.033408] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7530.979462] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7530.985128] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7530.987219] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7531.000637] Lustre: Skipped 2 previous similar messages [ 7541.219439] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7541.230130] Lustre: Skipped 4 previous similar messages [ 7544.287153] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7544.514798] Lustre: server umount lustre-OST0000 complete [ 7549.413101] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7549.421708] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7555.573554] Lustre: server umount lustre-MDT0000 complete [ 7556.576914] LustreError: 306280:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7556.624621] LustreError: 306280:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7559.830664] LustreError: 306709:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788745356 with bad export cookie 14044145969864516400 [ 7559.837990] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7559.845554] LustreError: 306709:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7560.550581] Lustre: server umount lustre-MDT0001 complete [ 7572.911704] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing unload_modules_local [ 7576.347675] Key type lgssc unregistered [ 7576.765618] LNet: 307929:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7576.776344] LNetError: 307929:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7576.794879] LNet: Removed LNI 192.168.203.129@tcp [ 7577.997423] Key type .llcrypt unregistered [ 7578.004415] Key type ._llcrypt unregistered [ 7601.219854] Key type ._llcrypt registered [ 7601.232660] Key type .llcrypt registered [ 7601.388927] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_hostid [ 7616.239822] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 7617.775047] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7617.873446] alg: No test for adler32 (adler32-zlib) [ 7619.076439] Lustre: Lustre: Build Version: 2.17.58_3_g671df26 [ 7619.366844] LNet: Added LNI 192.168.203.129@tcp [8/256/0/180] [ 7621.015349] Key type lgssc registered [ 7622.396918] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7648.445553] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 7663.667863] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7663.728882] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7665.196918] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7665.279959] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7665.450978] Lustre: lustre-MDT0000: new disk, initializing [ 7665.586634] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7665.624053] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7671.768107] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 7683.503859] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7683.973878] Lustre: lustre-OST0000: new disk, initializing [ 7683.979983] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7683.984367] Lustre: Skipped 1 previous similar message [ 7683.989975] Lustre: 311979:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7684.073731] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7691.433614] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 7692.541346] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 7692.564380] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 7692.609253] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 8511.975576] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8511.989963] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8512.781765] Lustre: server umount lustre-MDT0000 complete [ 8517.217496] LustreError: 311166:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788746314 with bad export cookie 11625901098584631976 [ 8517.223326] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8517.239926] LustreError: 311166:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8517.628852] Lustre: server umount lustre-OST0000 complete [ 8542.977965] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_hostid [ 8552.381059] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 8593.768666] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8594.015067] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8594.059781] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8594.164470] Lustre: lustre-MDT0000: new disk, initializing [ 8594.269683] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8594.308142] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8598.800816] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 8609.645552] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8609.796915] Lustre: 316313:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8609.874296] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8609.882436] Lustre: Skipped 1 previous similar message [ 8609.982323] Lustre: lustre-MDT0001: new disk, initializing [ 8610.082580] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8610.126190] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8610.137782] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8614.211179] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 8622.494915] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8627.118595] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8632.899674] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8633.102884] Lustre: lustre-OST0000: new disk, initializing [ 8633.105420] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8633.110391] Lustre: 318281:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8633.167379] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8640.969378] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 8641.640884] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8641.653509] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8641.758219] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8650.809606] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8655.658324] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 8655.910325] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 8660.352377] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 8660.629131] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 8667.625181] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8667.630806] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8667.633853] LustreError: Skipped 1 previous similar message [ 8667.646978] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8667.681986] Lustre: Skipped 1 previous similar message [ 8667.722764] Lustre: Skipped 1 previous similar message [ 8672.764658] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8677.343139] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8677.795468] Lustre: server umount lustre-OST0000 complete [ 8683.498918] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8683.522089] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8683.535783] Lustre: Skipped 2 previous similar messages [ 8688.610415] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8688.623398] Lustre: Skipped 1 previous similar message [ 8690.135463] Lustre: server umount lustre-MDT0000 complete [ 8694.995733] LustreError: 316305:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788746491 with bad export cookie 11625901098587572795 [ 8695.005478] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8695.009928] LustreError: 316305:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8695.069486] LustreError: 309255:0:(import.c:761:ptlrpc_connect_import_locked()) already connecting [ 8695.469860] Lustre: server umount lustre-MDT0001 complete [ 8699.440156] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 8701.072689] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 22:01:37 (1788746497) [ 8708.262474] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8708.719714] LustreError: 320604:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8708.741447] LustreError: 320604:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8708.795565] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8713.401616] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 8714.213062] LustreError: 320605:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8718.311739] LustreError: 320604:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8721.853728] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8722.252968] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8727.395043] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 8734.939456] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8740.436288] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8747.849304] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8748.109408] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8754.570843] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 8762.333357] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8769.127801] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8769.308474] Lustre: lustre-OST0001: new disk, initializing [ 8769.312927] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8769.319921] Lustre: 323441:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8769.407563] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8775.312504] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 8777.805250] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8777.812360] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8777.855102] LustreError: 323901:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 8778.914784] LustreError: 323906:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 8778.923287] LustreError: 323906:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 8779.942264] LustreError: 323901:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 8779.978632] LustreError: 323901:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 8781.992507] LustreError: 323906:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 8782.010136] LustreError: 323906:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 8784.985744] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 8785.147636] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 8786.149753] LustreError: 323906:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 8786.166102] LustreError: 323906:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 8790.512503] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff90e5c7b8a800.ost_server_uuid 50 [ 8791.906504] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff90e5c7b8a800.ost_server_uuid in FULL state after 0 sec [ 8794.471871] LustreError: 323906:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 8794.480863] LustreError: 323906:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 16 previous similar messages [ 8797.594247] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff90e5c7b8a800.ost_server_uuid 50 [ 8799.073707] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff90e5c7b8a800.ost_server_uuid in FULL state after 0 sec [ 8800.707163] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8816.100295] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8816.105768] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8816.110630] LustreError: Skipped 1 previous similar message [ 8816.124097] Lustre: Skipped 1 previous similar message [ 8816.125260] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 8816.163931] Lustre: Skipped 1 previous similar message [ 8820.572944] Lustre: server umount lustre-OST0001 complete [ 8821.304865] LustreError: 322428:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.203.29@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8829.412524] LustreError: 322428:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8829.417179] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8829.417978] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8829.431897] LustreError: 322428:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8829.464506] Lustre: Skipped 1 previous similar message [ 8832.287753] Lustre: server umount lustre-OST0000 complete [ 8839.651025] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8839.666184] Lustre: Skipped 1 previous similar message [ 8839.678914] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8839.684524] Lustre: Skipped 5 previous similar messages [ 8842.863125] Lustre: server umount lustre-MDT0000 complete [ 8844.772481] LustreError: 320601:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8844.794096] LustreError: 320601:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8846.924031] LustreError: 320585:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788746643 with bad export cookie 11625901098587573677 [ 8846.929059] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8846.938856] LustreError: 320585:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8847.302754] Lustre: server umount lustre-MDT0001 complete [ 8858.139951] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing unload_modules_local [ 8861.551757] Key type lgssc unregistered [ 8861.856867] LNet: 326128:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8861.874907] LNetError: 326128:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8861.889151] LNet: Removed LNI 192.168.203.129@tcp [ 8862.851824] Key type .llcrypt unregistered [ 8862.854191] Key type ._llcrypt unregistered [ 8888.490839] Key type ._llcrypt registered [ 8888.493689] Key type .llcrypt registered [ 8888.645707] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 8890.044147] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8890.104382] alg: No test for adler32 (adler32-zlib) [ 8891.196173] Lustre: Lustre: Build Version: 2.17.58_3_g671df26 [ 8891.513931] LNet: Added LNI 192.168.203.129@tcp [8/256/0/180] [ 8893.194503] Key type lgssc registered [ 8894.440594] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8901.830519] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8901.859780] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8903.324730] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8907.787744] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 8913.902568] LustreError: 327547:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8913.925246] LustreError: 327547:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8915.034903] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8919.011554] LustreError: 327546:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8921.846884] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8922.250306] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8926.739125] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 8937.045298] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8937.386650] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8943.270853] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 8947.637372] LustreError: 329230:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8948.655088] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 8950.666564] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8953.840852] LustreError: 329227:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8953.868865] LustreError: 329227:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8957.729403] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8958.038033] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8963.075944] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 8965.006696] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 8971.925210] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8977.857097] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 8978.138545] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9080.807577] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9080.821092] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9080.827871] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 9080.846734] Lustre: Skipped 1 previous similar message [ 9083.887289] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 9083.892932] Lustre: Skipped 2 previous similar messages [ 9086.107596] Lustre: server umount lustre-OST0001 complete [ 9088.997234] LustreError: 329230:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9089.022924] LustreError: 329230:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9094.126902] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9094.133817] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9094.150440] Lustre: Skipped 1 previous similar message [ 9098.413974] Lustre: server umount lustre-OST0000 complete [ 9104.352645] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9104.376505] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9104.380920] Lustre: Skipped 2 previous similar messages [ 9108.858188] Lustre: server umount lustre-MDT0000 complete [ 9109.475446] LustreError: 328470:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9109.496786] LustreError: 328470:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9112.588168] LustreError: 327527:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788746909 with bad export cookie 11901008467721377722 [ 9112.592401] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9112.605114] LustreError: 327527:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9112.987224] Lustre: server umount lustre-MDT0001 complete [ 9124.151495] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing unload_modules_local [ 9127.202186] Key type lgssc unregistered [ 9127.579648] LNet: 332488:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9127.591521] LNetError: 332488:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9127.614563] LNet: Removed LNI 192.168.203.129@tcp [ 9128.764639] Key type .llcrypt unregistered [ 9128.767648] Key type ._llcrypt unregistered [ 9173.621310] Key type ._llcrypt registered [ 9173.624539] Key type .llcrypt registered [ 9173.770635] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 9175.080632] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9175.113585] alg: No test for adler32 (adler32-zlib) [ 9176.344531] Lustre: Lustre: Build Version: 2.17.58_3_g671df26 [ 9176.744674] LNet: Added LNI 192.168.203.129@tcp [8/256/0/180] [ 9178.391204] Key type lgssc registered [ 9179.768395] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9188.374900] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9188.403645] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9189.850655] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9194.201426] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 9199.077241] LustreError: 334775:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9199.092786] LustreError: 334775:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 9202.057155] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9202.439072] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9206.753905] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 9209.568398] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 9216.422319] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9216.985235] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9222.816199] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 9227.243348] LustreError: 336263:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9231.083432] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9231.298865] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9234.385267] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 9234.385610] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 9237.301949] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 9245.765723] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9252.621810] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 9268.201876] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9268.213307] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9268.216866] Lustre: Skipped 3 previous similar messages [ 9268.227493] Lustre: Skipped 3 previous similar messages [ 9272.937461] Lustre: server umount lustre-MDT0000 complete [ 9273.317261] LustreError: 334769:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9273.335287] LustreError: 334769:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9276.780408] LustreError: 334756:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788747073 with bad export cookie 16649184914913109930 [ 9276.785865] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9276.795704] LustreError: 334756:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9277.135132] Lustre: server umount lustre-MDT0001 complete [ 9291.085997] Lustre: server umount lustre-OST0000 complete [ 9304.571321] Lustre: server umount lustre-OST0001 complete [ 9319.727481] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 9328.324746] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9328.677616] LustreError: 340044:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9328.746969] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9332.810703] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 9334.244218] LustreError: 340045:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9340.156834] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9340.454959] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9344.524706] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 9346.960578] Lustre: 341184:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 9353.157641] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9354.545112] LustreError: 341537:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9354.564430] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 9354.570424] LustreError: 341537:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9358.412479] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 9365.553254] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9365.742215] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9365.745844] Lustre: Skipped 1 previous similar message [ 9366.787038] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 9370.316527] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 9376.393451] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9379.251172] Lustre: 343053:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 9382.372955] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9382.374826] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9382.388800] Lustre: Skipped 1 previous similar message [ 9387.492499] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9387.496313] Lustre: Skipped 6 previous similar messages [ 9388.379774] Lustre: server umount lustre-MDT0000 complete [ 9391.484731] LustreError: 340025:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788747188 with bad export cookie 16649184914913111631 [ 9391.495320] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9391.503054] LustreError: 340025:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9391.751581] Lustre: server umount lustre-MDT0001 complete [ 9404.311048] Lustre: server umount lustre-OST0000 complete [ 9408.993521] Lustre: 334039:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788747189/real 1788747189] req@ffff898db593f480 x1875637146069248/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788747205 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 9409.030104] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9409.052518] Lustre: Skipped 2 previous similar messages [ 9409.799388] Lustre: server umount lustre-OST0001 complete [ 9416.084241] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 9418.509504] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 9420.659569] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 22:13:36 (1788747216) [ 9455.386898] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 9466.210691] loop0: detected capacity change from 0 to 1099511627776 [ 9467.055078] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 9467.063407] *** VALIDATE xfs *** [ 9467.096071] XFS (loop0): Mounting V5 Filesystem [ 9467.182338] XFS (loop0): Ending clean mount [ 9468.723442] loop1: detected capacity change from 0 to 18691697672192 [ 9574.332567] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_hostid [ 9583.489075] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 9617.350610] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9617.572029] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9617.598553] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9617.658590] Lustre: lustre-MDT0000: new disk, initializing [ 9617.754883] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9617.777655] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9621.820178] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 9631.042755] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9631.173559] Lustre: 348209:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9631.214834] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9631.218736] Lustre: Skipped 1 previous similar message [ 9631.274642] Lustre: lustre-MDT0001: new disk, initializing [ 9631.394328] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9631.400708] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9635.638602] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 9644.150739] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9650.653327] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9656.847576] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9657.087509] Lustre: lustre-OST0000: new disk, initializing [ 9657.090830] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9657.098560] Lustre: 350178:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9658.767884] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9658.780208] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9658.868662] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9662.786438] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 9671.417037] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9675.691283] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9675.915114] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9679.638473] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 9679.861171] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9684.482985] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9684.484800] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9684.492487] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9684.517845] Lustre: Skipped 1 previous similar message [ 9684.537899] Lustre: Skipped 1 previous similar message [ 9689.579815] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9689.591328] Lustre: Skipped 1 previous similar message [ 9696.223205] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9696.529333] Lustre: server umount lustre-OST0000 complete [ 9702.880678] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9702.891186] LustreError: Skipped 1 previous similar message [ 9702.903145] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9702.922044] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9702.934096] Lustre: Skipped 2 previous similar messages [ 9706.390244] Lustre: server umount lustre-MDT0000 complete [ 9709.895469] LustreError: 348203:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788747506 with bad export cookie 16649184914913113178 [ 9709.896893] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9709.908749] LustreError: 348203:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9710.196597] Lustre: server umount lustre-MDT0001 complete [ 9718.298738] XFS (loop0): Unmounting Filesystem [ 9722.419443] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 22:18:38 (1788747518) [ 9727.883773] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9728.252662] LustreError: 352736:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9728.270654] LustreError: 352736:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 9728.347958] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9728.356428] Lustre: Skipped 2 previous similar messages [ 9732.651763] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 9743.026295] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9748.683047] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 9758.220460] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9764.710302] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9771.869918] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9779.344662] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 9787.540812] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9799.139478] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9799.141323] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9799.155405] LustreError: Skipped 1 previous similar message [ 9799.157857] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9799.157868] Lustre: Skipped 2 previous similar messages [ 9799.186652] Lustre: Skipped 2 previous similar messages [ 9804.901734] Lustre: server umount lustre-OST0000 complete [ 9814.762973] Lustre: server umount lustre-MDT0000 complete [ 9818.599191] LustreError: 353472:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788747615 with bad export cookie 16649184914913114060 [ 9818.609320] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9818.617718] LustreError: 353472:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9819.243324] Lustre: server umount lustre-MDT0001 complete [ 9830.712617] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing unload_modules_local [ 9833.932945] Key type lgssc unregistered [ 9834.359769] LNet: 356365:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9834.383080] LNetError: 356365:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9834.414181] LNet: Removed LNI 192.168.203.129@tcp [ 9835.353544] Key type .llcrypt unregistered [ 9835.356470] Key type ._llcrypt unregistered [ 9860.718856] Key type ._llcrypt registered [ 9860.722647] Key type .llcrypt registered [ 9860.849829] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [ 9862.154751] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9862.191238] alg: No test for adler32 (adler32-zlib) [ 9863.312707] Lustre: Lustre: Build Version: 2.17.58_3_g671df26 [ 9863.531883] LNet: Added LNI 192.168.203.129@tcp [8/256/0/180] [ 9865.183177] Key type lgssc registered [ 9866.310798] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9873.875613] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9873.910731] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9875.476153] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9879.801616] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 9885.670717] LustreError: 357928:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9885.701726] LustreError: 357928:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 9887.463354] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9887.912064] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9891.825043] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 9898.113820] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9903.139251] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9909.917893] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9910.378148] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9916.795744] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 9925.227221] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9934.306237] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 9934.318409] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9934.334729] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9936.358629] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9936.364320] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9938.511590] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.29@tcp (stopping) [ 9938.518373] Lustre: Skipped 1 previous similar message [ 9940.037664] Lustre: server umount lustre-MDT0001 complete [ 9941.477964] LustreError: 357928:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9941.504518] LustreError: 357928:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9943.640482] LustreError: 360420:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.29@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9946.597075] LustreError: 358658:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9946.671842] LustreError: 358658:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9951.732870] LustreError: 357925:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9951.771386] LustreError: 357925:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 9952.880115] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9953.166996] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 9953.179156] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9953.240518] Lustre: 357918:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9953.751195] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9958.891249] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 9959.395397] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [ 9963.516165] LustreError: 361530:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 9963.530321] LustreError: 361530:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 9963.550392] LustreError: 357922:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 9963.669938] LustreError: 361533:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.203.129@tcp: cfg command failed: rc = -17 [ 9963.678472] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 9963.678472] [10266.084582] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [10266.087446] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10266.120408] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10266.123636] Lustre: Skipped 1 previous similar message [10271.204166] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10271.220227] Lustre: Skipped 2 previous similar messages [10271.893323] Lustre: server umount lustre-MDT0001 complete [10276.335919] LustreError: 358658:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10276.366378] LustreError: 358658:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10281.122564] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10281.208527] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [10281.215604] Lustre: Found index 1 for lustre-MDT0001, updating log [10281.246971] Lustre: 357920:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10281.545717] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [10285.580238] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [10286.613334] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [10290.161687] LustreError: 370164:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [10290.166725] LustreError: 370164:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [10290.172157] LustreError: 357922:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [10290.177504] LustreError: 357922:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [10290.222612] LustreError: 370167:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [10290.235123] LustreError: 370167:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [10290.241860] LustreError: 370167:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.203.129@tcp: cfg command failed: rc = -17 [10290.261280] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.203.129@tcp [10290.261280] [10594.279327] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [10594.288684] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10594.308486] Lustre: Skipped 1 previous similar message [10594.312601] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10594.326184] Lustre: Skipped 1 previous similar message [10596.729283] Lustre: server umount lustre-MDT0001 complete [10599.490750] LustreError: 357923:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.29@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10599.520184] LustreError: 357923:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10608.825033] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10609.025683] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [10609.029320] Lustre: Found index 1 for lustre-MDT0001, updating log [10609.162898] Lustre: 357918:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10609.629280] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [10614.775546] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [10615.041471] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [10616.365257] LustreError: 378839:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [10616.384822] LustreError: 378839:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [10616.398129] LustreError: 357922:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [10616.407252] LustreError: 357922:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [10616.509593] LustreError: 378846:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [10616.529860] LustreError: 378846:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.203.129@tcp: cfg command failed: rc = -114 [10616.543199] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [10616.543199] [10937.312849] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10937.316885] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10937.338078] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10937.355146] Lustre: Skipped 2 previous similar messages [10937.366909] Lustre: Skipped 2 previous similar messages [10940.807928] Lustre: server umount lustre-MDT0000 complete [10942.433732] LustreError: 357925:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10942.443357] LustreError: 357925:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [10944.475193] LustreError: 357912:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788748741 with bad export cookie 8620038195473725977 [10944.478486] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10944.488984] LustreError: 357912:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10947.561619] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10947.576396] Lustre: Skipped 1 previous similar message [10950.954110] Lustre: server umount lustre-MDT0001 complete [10954.965254] Lustre: server umount lustre-OST0000 complete [10964.289351] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_hostid [10973.842711] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [11009.319338] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 22:40:05 (1788748805) [11015.494535] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11015.682879] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11015.721949] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11015.809897] Lustre: lustre-MDT0000: new disk, initializing [11015.900284] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11015.925814] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11019.602509] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [11028.489964] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11028.607167] Lustre: 389859:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11028.671578] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11028.676050] Lustre: Skipped 1 previous similar message [11028.733340] Lustre: lustre-MDT0001: new disk, initializing [11028.799468] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11028.843504] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11028.857778] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11033.168981] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [11042.126481] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11046.958752] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11052.358465] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11052.610166] Lustre: lustre-OST0000: new disk, initializing [11052.613441] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11052.623860] Lustre: 391823:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11052.698071] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11058.772761] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_default_debug -1 all [11060.275299] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11060.288950] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11060.362655] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11066.917116] Lustre: DEBUG MARKER: oleg329-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11075.044744] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11075.051202] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11075.062278] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11075.069919] Lustre: Skipped 3 previous similar messages [11079.795266] Lustre: server umount lustre-MDT0000 complete [11080.677729] LustreError: 389863:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11080.695903] LustreError: 389863:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [11082.813272] LustreError: 389851:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788748879 with bad export cookie 8620038195473727930 [11082.819867] LustreError: MGC192.168.203.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11082.828015] LustreError: 389851:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11083.148389] Lustre: server umount lustre-MDT0001 complete [11097.044885] Lustre: server umount lustre-OST0000 complete [11120.920459] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing set_hostid [11128.499675] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing load_modules_local [11152.637666] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 11031 sec ========= 22:42:28 (1788748948) [11153.839206] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 22:42:30 (1788748950) === [11156.374601] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 22:42:32 (1788748952) === [11185.289589] Lustre: DEBUG MARKER: oleg329-server.virtnet: executing unload_modules_local [11187.125578] Key type lgssc unregistered [11187.386147] LNet: 398184:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11187.398845] LNetError: 398184:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11187.416902] LNet: Removed LNI 192.168.203.129@tcp [11188.056458] Key type .llcrypt unregistered [11188.062336] Key type ._llcrypt unregistered