[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 529327866 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001017] APIC: Switch to symmetric I/O mode setup [ 0.003366] x2apic enabled [ 0.004015] Switched APIC routing to physical x2apic. [ 0.005020] kvm-guest: setup PV IPIs [ 0.008000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008038] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009020] pid_max: default: 32768 minimum: 301 [ 0.011196] LSM: Security Framework initializing [ 0.012077] Yama: becoming mindful. [ 0.013066] SELinux: Initializing. [ 0.014097] *** VALIDATE selinux *** [ 0.023222] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.028319] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.029182] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030129] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031160] *** VALIDATE tmpfs *** [ 0.033386] *** VALIDATE proc *** [ 0.035098] *** VALIDATE cgroup *** [ 0.036016] *** VALIDATE cgroup2 *** [ 0.038068] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.039193] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.040016] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.041041] Spectre V2 : User space: Vulnerable [ 0.042017] Speculative Store Bypass: Vulnerable [ 0.045506] debug: unmapping init [mem 0xffffffff9ee59000-0xffffffff9ee60fff] [ 0.047973] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.048795] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.049036] ... version: 2 [ 0.050020] ... bit width: 48 [ 0.051016] ... generic registers: 4 [ 0.052019] ... value mask: 0000ffffffffffff [ 0.053021] ... max period: 00007fffffffffff [ 0.054022] ... fixed-purpose events: 3 [ 0.055019] ... event mask: 000000070000000f [ 0.056393] rcu: Hierarchical SRCU implementation. [ 0.058775] smp: Bringing up secondary CPUs ... [ 0.059719] x86: Booting SMP configuration: [ 0.060039] .... node #0, CPUs: #1 #2 #3 [ 0.064119] smp: Brought up 1 node, 4 CPUs [ 0.066025] smpboot: Max logical packages: 1 [ 0.067019] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.241406] node 0 deferred pages initialised in 170ms [ 0.245039] devtmpfs: initialized [ 0.246298] x86/mm: Memory block size: 128MB [ 0.249837] gcov: version magic: 0x41383552 [ 0.253200] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.254081] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.255328] pinctrl core: initialized pinctrl subsystem [ 0.257182] [ 0.257758] ************************************************************* [ 0.260027] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.262009] ** ** [ 0.263009] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.265013] ** ** [ 0.267015] ** This means that this kernel is built to expose internal ** [ 0.269011] ** IOMMU data structures, which may compromise security on ** [ 0.271012] ** your system. ** [ 0.273015] ** ** [ 0.276015] ** If you see this message and you are not debugging the ** [ 0.278014] ** kernel, report this immediately to your vendor! ** [ 0.281015] ** ** [ 0.283013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.285023] ************************************************************* [ 0.287734] NET: Registered protocol family 16 [ 0.290547] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.293049] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.294061] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.298022] cpuidle: using governor menu [ 0.299529] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.301481] PCI: Using configuration type 1 for base access [ 0.304138] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.313033] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.314010] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.316136] cryptd: max_cpu_qlen set to 1000 [ 0.318173] ACPI: Added _OSI(Module Device) [ 0.320012] ACPI: Added _OSI(Processor Device) [ 0.321011] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.322009] ACPI: Added _OSI(Processor Aggregator Device) [ 0.326363] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.332292] ACPI: Interpreter enabled [ 0.333052] ACPI: PM: (supports S0 S3 S4 S5) [ 0.334015] ACPI: Using IOAPIC for interrupt routing [ 0.336093] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.338412] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.347787] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.350062] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.353023] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.355069] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.359825] acpiphp: Slot [2] registered [ 0.361135] acpiphp: Slot [5] registered [ 0.363122] acpiphp: Slot [6] registered [ 0.364133] acpiphp: Slot [7] registered [ 0.366120] acpiphp: Slot [8] registered [ 0.367176] acpiphp: Slot [9] registered [ 0.369114] acpiphp: Slot [10] registered [ 0.370120] acpiphp: Slot [3] registered [ 0.372128] acpiphp: Slot [4] registered [ 0.374135] acpiphp: Slot [11] registered [ 0.375113] acpiphp: Slot [12] registered [ 0.377171] acpiphp: Slot [13] registered [ 0.378099] acpiphp: Slot [14] registered [ 0.380095] acpiphp: Slot [15] registered [ 0.381094] acpiphp: Slot [16] registered [ 0.383138] acpiphp: Slot [17] registered [ 0.384107] acpiphp: Slot [18] registered [ 0.386116] acpiphp: Slot [19] registered [ 0.387131] acpiphp: Slot [20] registered [ 0.389162] acpiphp: Slot [21] registered [ 0.391082] acpiphp: Slot [22] registered [ 0.392148] acpiphp: Slot [23] registered [ 0.393091] acpiphp: Slot [24] registered [ 0.395105] acpiphp: Slot [25] registered [ 0.397179] acpiphp: Slot [26] registered [ 0.398130] acpiphp: Slot [27] registered [ 0.400106] acpiphp: Slot [28] registered [ 0.401197] acpiphp: Slot [29] registered [ 0.403113] acpiphp: Slot [30] registered [ 0.405147] acpiphp: Slot [31] registered [ 0.406089] PCI host bridge to bus 0000:00 [ 0.408021] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.410025] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.413039] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.416028] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.419030] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.421041] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.423192] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.427096] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.430383] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.439828] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.445505] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.448022] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.450022] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.453031] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.456218] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.457538] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.459026] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.461612] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.465020] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.475033] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.479017] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.482310] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.488020] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.495027] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.511019] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.520351] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.531020] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.542022] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.563018] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.571026] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.583017] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.592018] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.616025] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.627922] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.645020] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.657018] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.671027] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.677482] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.683019] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.689029] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.703018] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.712897] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.728023] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.734018] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.769022] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.783743] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.785365] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.787342] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.789285] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.792324] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.797122] iommu: Default domain type: Passthrough [ 0.799540] SCSI subsystem initialized [ 0.801153] ACPI: bus type USB registered [ 0.803150] usbcore: registered new interface driver usbfs [ 0.804062] usbcore: registered new interface driver hub [ 0.806096] usbcore: registered new device driver usb [ 0.807237] pps_core: LinuxPPS API ver. 1 registered [ 0.809009] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.811062] PTP clock support registered [ 0.813147] EDAC MC: Ver: 3.0.0 [ 0.814248] PCI: Using ACPI for IRQ routing [ 0.816277] NetLabel: Initializing [ 0.818017] NetLabel: domain hash size = 128 [ 0.820015] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.822113] NetLabel: unlabeled traffic allowed by default [ 0.825277] vgaarb: loaded [ 0.827313] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.829020] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.835311] clocksource: Switched to clocksource kvm-clock [ 0.934376] VFS: Disk quotas dquot_6.6.0 [ 0.935479] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.937195] *** VALIDATE ramfs *** [ 0.938268] *** VALIDATE hugetlbfs *** [ 0.939339] pnp: PnP ACPI init [ 0.941796] pnp: PnP ACPI: found 6 devices [ 0.956775] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.959380] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.961272] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.963155] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.964928] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.966774] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.968853] NET: Registered protocol family 2 [ 0.970926] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.974875] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.977686] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.981860] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.986721] TCP: Hash tables configured (established 65536 bind 65536) [ 0.989711] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.992269] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.994698] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.997318] NET: Registered protocol family 1 [ 1.000122] RPC: Registered named UNIX socket transport module. [ 1.002582] RPC: Registered udp transport module. [ 1.004271] RPC: Registered tcp transport module. [ 1.006184] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.008983] NET: Registered protocol family 44 [ 1.011087] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.013543] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.016133] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.018731] PCI: CLS 0 bytes, default 64 [ 1.021408] Unpacking initramfs... [ 2.481977] debug: unmapping init [mem 0xffff9b213cc54000-0xffff9b213ffbffff] [ 2.486393] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.489144] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.492482] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.011488] Initialise system trusted keyrings [ 3.012660] Key type blacklist registered [ 3.014110] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.025359] zbud: loaded [ 3.029435] *** VALIDATE nfs *** [ 3.031753] *** VALIDATE nfs4 *** [ 3.033437] pstore: using deflate compression [ 3.036821] Platform Keyring initialized [ 3.145840] NET: Registered protocol family 38 [ 3.147187] Key type asymmetric registered [ 3.148395] Asymmetric key parser 'x509' registered [ 3.149833] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.151769] io scheduler mq-deadline registered [ 3.152955] io scheduler kyber registered [ 3.154274] io scheduler bfq registered [ 3.155647] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.157687] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.159557] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.161579] ACPI: Power Button [PWRF] [ 3.165564] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.170125] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.179901] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.185527] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.197426] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.223065] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.249052] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.253739] Non-volatile memory driver v1.3 [ 3.254800] Linux agpgart interface v0.103 [ 3.285154] virtio_blk virtio1: [vda] 145904 512-byte logical blocks (74.7 MB/71.2 MiB) [ 3.288287] vda: detected capacity change from 0 to 74702848 [ 3.302626] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.306023] vdb: detected capacity change from 0 to 1073741824 [ 3.323370] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.326748] vdc: detected capacity change from 0 to 2621440000 [ 3.349691] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.352742] vdd: detected capacity change from 0 to 2621440000 [ 3.372268] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.375251] vde: detected capacity change from 0 to 4294967296 [ 3.393172] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.397068] vdf: detected capacity change from 0 to 4294967296 [ 3.405265] libphy: Fixed MDIO Bus: probed [ 3.411829] usbcore: registered new interface driver usbserial_generic [ 3.413985] usbserial: USB Serial support registered for generic [ 3.416151] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.420528] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.422618] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.424724] mousedev: PS/2 mouse device common for all mice [ 3.427287] rtc_cmos 00:05: RTC can wake from S4 [ 3.429686] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.433841] rtc_cmos 00:05: registered as rtc0 [ 3.436171] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.436404] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.441423] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.443856] intel_pstate: CPU model not supported [ 3.452355] hid: raw HID events driver (C) Jiri Kosina [ 3.454554] usbcore: registered new interface driver usbhid [ 3.456554] usbhid: USB HID core driver [ 3.458159] drop_monitor: Initializing network drop monitor service [ 3.460557] Initializing XFRM netlink socket [ 3.462407] NET: Registered protocol family 10 [ 3.465255] Segment Routing with IPv6 [ 3.466755] NET: Registered protocol family 17 [ 3.468816] mpls_gso: MPLS GSO support [ 3.475124] RAS: Correctable Errors collector initialized. [ 3.477349] AVX version of gcm_enc/dec engaged. [ 3.478963] AES CTR mode by8 optimization enabled [ 3.557461] sched_clock: Marking stable (3557414049, 0)->(4625845860, -1068431811) [ 3.561511] registered taskstats version 1 [ 3.563410] Loading compiled-in X.509 certificates [ 3.565468] zswap: loaded using pool lzo/zbud [ 3.592528] Key type big_key registered [ 3.604265] Key type encrypted registered [ 3.605738] ima: No TPM chip found, activating TPM-bypass! [ 3.607251] ima: Allocated hash algorithm: sha1 [ 3.609090] ima: No architecture policies found [ 3.610735] evm: Initialising EVM extended attributes: [ 3.612371] evm: security.selinux [ 3.613603] evm: security.ima [ 3.614672] evm: security.capability [ 3.615922] evm: HMAC attrs: 0x1 [ 3.618283] rtc_cmos 00:05: setting system clock to 2026-08-14 15:14:22 UTC (1786720462) [ 3.624419] debug: unmapping init [mem 0xffffffff9fe03000-0xffffffff9fffffff] [ 3.627483] debug: unmapping init [mem 0xffffffff9eb82000-0xffffffff9ee58fff] [ 3.636079] Write protecting the kernel read-only data: 28672k [ 3.639593] debug: unmapping init [mem 0xffffffff9d203000-0xffffffff9d3fffff] [ 3.642588] debug: unmapping init [mem 0xffffffff9db14000-0xffffffff9dbfffff] [ 3.680166] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.688253] systemd[1]: Detected virtualization kvm. [ 3.689819] systemd[1]: Detected architecture x86-64. [ 3.691404] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.714757] systemd[1]: No hostname configured. [ 3.716683] systemd[1]: Set hostname to . [ 3.718905] random: systemd: uninitialized urandom read (16 bytes read) [ 3.721414] systemd[1]: Initializing machine ID from random generator. [ 3.863988] random: systemd: uninitialized urandom read (16 bytes read) [ 3.865667] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 3.868162] random: systemd: uninitialized urandom read (16 bytes read) [ 3.869635] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ 3.872266] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Listening on udev Kernel Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Local File Systems. [ OK ] Listening on udev Control Socket. [ OK ] Listening on Journal Socket. Starting Apply Kernel Variables... Starting Journal Service... Starting Setup Virtual Console... [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Sockets. Starting Create Volatile Files and Directories... [ OK ] Reached target Swap. [ OK ] Reached target Initrd Root Device. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.502638] device-mapper: uevent: version 1.0.3 [ 4.506456] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.271954] random: fast init done [ 5.274949] virtio_net virtio0 ens2: renamed from eth0 [ 5.316926] scsi host0: ata_piix [ 5.354987] scsi host1: ata_piix [ 5.356288] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.358093] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.643476] dracut-initqueue[590]: RTNETLINK answers: File exists [ 10.121644] random: crng init done [ 10.123081] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.513108] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Swap. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.602590] printk: systemd: 24 output lines suppressed due to ratelimiting [ 11.876061] SELinux: Disabled at runtime. [ 11.936526] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.946307] systemd[1]: Detected virtualization kvm. [ 11.948638] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.433754] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.437129] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.441057] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.443705] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.445736] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.452468] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.461068] systemd[1]: Listening on RPCbind Server Activation Socket. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Listening on Process Core Dump Socket. [ OK ] Stopped target Switch Root. [ OK ] Created slice User and Session Slice. [ OK ] Stopped target Initrd File Systems. Starting Remount Root and Kernel File Systems... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Stopped target Initrd Root File System. [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice system-sshd\x2dkeygen.slice. Mounting Kernel Debug File System... [ OK ] Listening on udev Control Socket. [ OK ] Reached target rpc_pipefs.target. [ OK ] Created slice system-getty.slice. Mounting Huge Pages File System... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Reached target Slices. [ OK ] Created slice system-serial\x2dgetty.slice. Mounting POSIX Message Queue File System... Starting Apply Kernel Variables... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... Activating swap /dev/disk/by-label/SWAP... [ OK ] Started Journal Service. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ 12.635767] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Configure read-only root support... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 12.821772] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Coldplug all Devices. [ OK ] Started udev Kernel Device Manager. [ 13.166994] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.226373] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.357096] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.383561] EDAC sbridge: Ver: 1.1.2 [ 15.163861] Key type dns_resolver registered [ 15.519334] NFS: Registering the id_resolver key type [ 15.521934] Key type id_resolver registered [ 15.523535] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Load/Save Random Seed. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... Starting Login Service... [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... [ OK ] Started OpenSSH server daemon. Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg234-server login: [ 41.679993] libcfs: loading out-of-tree module taints kernel. [ 41.700069] Key type ._llcrypt registered [ 41.701777] Key type .llcrypt registered [ 41.754463] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 50.287812] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 50.909265] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 50.916512] alg: No test for adler32 (adler32-zlib) [ 52.334535] Lustre: Lustre: Build Version: 2.17.57_1_g689e325 [ 53.357886] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 55.239233] Key type lgssc registered [ 57.210547] Lustre: Echo OBD driver; http://www.lustre.org/ [ 71.876251] hrtimer: interrupt took 3111636 ns [ 79.003331] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 130.436874] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 146.391721] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 146.441657] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 148.027173] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 148.059497] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 148.335762] Lustre: lustre-MDT0000: new disk, initializing [ 148.533144] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 148.577112] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 153.975539] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 169.093715] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 169.200312] Lustre: 6494:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 169.270955] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 169.277365] Lustre: Skipped 1 previous similar message [ 169.449318] Lustre: lustre-MDT0001: new disk, initializing [ 169.575682] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 169.615831] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 169.636226] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 174.701765] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 179.701451] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 190.236043] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 190.401633] Lustre: lustre-OST0000: new disk, initializing [ 190.408047] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 190.414506] Lustre: 8431:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 190.492537] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 195.113961] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 195.128457] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 195.209246] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 197.552322] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 213.419578] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 213.590288] Lustre: lustre-OST0001: new disk, initializing [ 213.603163] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 213.618048] Lustre: 9506:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 213.705559] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 219.733440] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 219.743007] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 219.883141] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 221.943566] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 235.907344] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 244.359492] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 253.020199] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing check_logdir /tmp/testlogs/ [ 258.933678] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing yml_node [ 263.902433] Lustre: DEBUG MARKER: Client: 2.17.57.1 [ 266.861047] Lustre: DEBUG MARKER: MDS: 2.17.57.1 [ 270.093614] Lustre: DEBUG MARKER: OSS: 2.17.57.1 [ 272.172553] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Fri Aug 14 11:18:48 EDT 2026 [ 292.846542] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 294.762398] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 312.301150] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 312.317583] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 317.025502] Lustre: server umount lustre-MDT0000 complete [ 321.264822] LustreError: 6486:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786720780 with bad export cookie 10311082023372162461 [ 321.266855] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 321.285605] LustreError: 6486:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 322.530065] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 322.530815] LustreError: 6502:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 322.531347] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 322.531352] Lustre: Skipped 3 previous similar messages [ 322.555043] Lustre: Skipped 4 previous similar messages [ 322.596260] LustreError: 6502:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 326.561360] LustreError: 10083:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 326.567104] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 326.577625] LustreError: 10083:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 326.591292] Lustre: Skipped 2 previous similar messages [ 327.652685] Lustre: server umount lustre-MDT0001 complete [ 331.801574] Lustre: server umount lustre-OST0000 complete [ 336.284739] Lustre: server umount lustre-OST0001 complete [ 344.801213] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 355.519970] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 399.701884] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 400.187188] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 400.263610] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 400.370730] Lustre: lustre-MDT0000: new disk, initializing [ 400.585488] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 400.653972] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 405.807341] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 417.987539] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 418.121695] Lustre: 15065:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 418.195236] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 418.200367] Lustre: Skipped 1 previous similar message [ 418.300550] Lustre: lustre-MDT0001: new disk, initializing [ 418.455643] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 418.500639] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 418.526851] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 424.598673] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 437.118793] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 444.359865] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 451.736088] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 452.023408] Lustre: lustre-OST0000: new disk, initializing [ 452.027980] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 452.038468] Lustre: 17034:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 452.136836] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 453.474417] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 453.483661] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 453.615456] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 459.237278] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 470.079262] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 475.348908] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 475.607232] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 480.339206] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 480.549954] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 484.323126] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 484.329925] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 484.332425] LustreError: Skipped 1 previous similar message [ 484.362242] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 484.392037] Lustre: Skipped 1 previous similar message [ 488.774828] Lustre: server umount lustre-OST0000 complete [ 494.563309] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 494.582779] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 494.589485] Lustre: Skipped 1 previous similar message [ 504.801815] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 504.811457] Lustre: Skipped 3 previous similar messages [ 507.359154] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 507.862382] Lustre: server umount lustre-MDT0000 complete [ 509.922500] LustreError: 15075:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 509.946256] LustreError: 15075:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 511.970768] LustreError: 17033:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786720970 with bad export cookie 10311082023372163994 [ 511.977811] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 511.982549] LustreError: 17033:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 512.298983] Lustre: server umount lustre-MDT0001 complete [ 517.703234] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 11:22:54 (1786720974) [ 525.039281] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 525.844477] LustreError: 19300:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 525.912633] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 530.925712] LustreError: 19301:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 531.475690] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 542.183331] LustreError: 19301:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 542.221309] LustreError: 19301:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 543.256795] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 550.174847] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 559.715960] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 566.987084] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 575.486959] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 582.137878] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 590.249542] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 605.434901] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 612.337152] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 612.343471] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 612.358993] Lustre: Skipped 3 previous similar messages [ 612.388301] Lustre: Skipped 2 previous similar messages [ 614.515710] Lustre: server umount lustre-MDT0000 complete [ 617.445737] LustreError: 20029:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 617.461943] LustreError: 20029:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 619.329479] LustreError: 19280:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786721078 with bad export cookie 10311082023372164876 [ 619.346317] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 619.359941] LustreError: 19280:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 620.248386] Lustre: server umount lustre-MDT0001 complete [ 636.356733] Lustre: server umount lustre-OST0000 complete [ 662.033860] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 673.038723] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 673.630994] LustreError: 23707:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 673.728484] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 673.749392] Lustre: Skipped 2 previous similar messages [ 678.905167] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 689.008654] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 694.334594] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 697.774691] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 705.313105] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 713.258701] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 722.692938] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 722.869500] Lustre: lustre-OST0001: new disk, initializing [ 722.873634] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 722.887605] Lustre: 26023:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 724.238457] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 724.258837] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 724.466696] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 729.502364] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 740.100529] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 743.916854] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 756.192435] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 756.198449] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 756.222378] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 761.612313] Lustre: server umount lustre-MDT0000 complete [ 765.430887] LustreError: 23704:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 765.460869] LustreError: 23704:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 766.068043] LustreError: 24447:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786721224 with bad export cookie 10311082023372165723 [ 766.071055] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 766.085217] LustreError: 24447:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 766.573810] Lustre: server umount lustre-MDT0001 complete [ 782.043252] Lustre: server umount lustre-OST0000 complete [ 796.924852] Lustre: server umount lustre-OST0001 complete [ 815.683548] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 826.206656] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 826.719397] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 826.736989] Lustre: Skipped 3 previous similar messages [ 831.102939] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 841.261331] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 846.590580] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 849.944760] Lustre: 30191:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 856.869073] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 863.584310] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 871.718311] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 879.007619] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 886.401965] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 890.632695] Lustre: 32069:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 897.508233] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 897.521294] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 897.535866] Lustre: Skipped 5 previous similar messages [ 897.546707] Lustre: Skipped 7 previous similar messages [ 903.339674] Lustre: server umount lustre-MDT0000 complete [ 907.206815] LustreError: 29025:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786721366 with bad export cookie 10311082023372167018 [ 907.227081] LustreError: 29025:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 907.230403] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 907.641812] Lustre: server umount lustre-MDT0001 complete [ 922.881321] Lustre: server umount lustre-OST0000 complete [ 924.127276] Lustre: 3623:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786721366/real 1786721366] req@ffff9b21be667100 x1873512247895168/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786721382 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 927.102392] Lustre: server umount lustre-OST0001 complete [ 937.713094] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 11:29:54 (1786721394) [ 965.996607] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 976.421582] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 976.764655] LustreError: 34877:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 976.786716] LustreError: 34877:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 980.866987] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 989.125089] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 995.435050] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 999.361360] Lustre: 36025:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1007.124529] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1014.515449] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1024.021808] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1031.279865] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1039.138941] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1042.898534] Lustre: 37903:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1050.085118] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1050.089823] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1050.098504] Lustre: Skipped 3 previous similar messages [ 1050.109520] Lustre: Skipped 7 previous similar messages [ 1054.329211] Lustre: server umount lustre-MDT0000 complete [ 1057.681937] LustreError: 34858:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786721516 with bad export cookie 10311082023372168124 [ 1057.688505] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1057.692306] LustreError: 34858:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1057.999466] Lustre: server umount lustre-MDT0001 complete [ 1072.034764] Lustre: server umount lustre-OST0000 complete [ 1086.585157] Lustre: server umount lustre-OST0001 complete [ 1096.485758] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 11:32:33 (1786721553) [ 1119.124437] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 1131.965667] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1132.869348] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1132.885744] Lustre: Skipped 7 previous similar messages [ 1140.224674] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1149.240958] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1155.249589] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1158.318652] Lustre: 41219:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1166.958804] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1174.722938] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1183.674342] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1189.790440] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1198.023677] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1201.907243] Lustre: 43098:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1202.857957] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1202.862508] Lustre: Skipped 1 previous similar message [ 1205.728267] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1205.734455] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1205.750330] Lustre: Skipped 2 previous similar messages [ 1212.206854] Lustre: server umount lustre-MDT0000 complete [ 1216.380099] LustreError: 40053:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786721675 with bad export cookie 10311082023372169230 [ 1216.388981] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1216.391908] LustreError: 40053:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 1223.067666] Lustre: server umount lustre-MDT0001 complete [ 1232.116061] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1241.568149] LustreError: 3622:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff9b21bfa5ce00 x1873512248000256/t0(0) o250->MGC192.168.202.134@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1241.958569] LustreError: 41583:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1241.976970] LustreError: 41583:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 29 previous similar messages [ 1247.322854] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1252.342368] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1256.960933] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1262.464766] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1262.595802] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1262.603320] Lustre: Skipped 1 previous similar message [ 1270.409967] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1272.010957] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1277.911930] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1279.706577] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1288.223335] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1290.610871] Lustre: server umount lustre-MDT0000 complete [ 1294.546200] LustreError: 41604:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786721753 with bad export cookie 10311082023372170357 [ 1294.559606] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1294.567442] LustreError: 41604:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1294.971464] Lustre: server umount lustre-MDT0001 complete [ 1309.189759] Lustre: server umount lustre-OST0000 complete [ 1322.693314] Lustre: server umount lustre-OST0001 complete [ 1331.401288] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 11:36:28 (1786721788) [ 1349.153676] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 1358.216757] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1362.128464] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1370.020518] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1374.823764] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1377.616464] Lustre: 48858:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1383.608499] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1389.986607] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1400.966026] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1410.938632] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1419.302550] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1424.445623] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1437.153609] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1437.165281] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1437.189687] Lustre: Skipped 9 previous similar messages [ 1437.200963] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1437.212395] Lustre: Skipped 15 previous similar messages [ 1437.561425] Lustre: server umount lustre-MDT0000 complete [ 1441.592511] LustreError: 50855:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786721900 with bad export cookie 10311082023372171148 [ 1441.597193] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1441.610111] LustreError: 50855:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1441.905223] Lustre: server umount lustre-MDT0001 complete [ 1456.196285] Lustre: server umount lustre-OST0000 complete [ 1458.147161] Lustre: 3624:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786721901/real 1786721901] req@ffff9b2085c82680 x1873512248092160/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786721917 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1460.387616] Lustre: server umount lustre-OST0001 complete [ 1472.123135] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 11:38:48 (1786721928) [ 1475.238099] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1477.616486] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 11:38:54 (1786721934) [ 1490.567358] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 1499.287078] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 1524.034346] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1524.256721] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1524.334736] Lustre: lustre-MDT0000: new disk, initializing [ 1524.456266] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1530.150932] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1540.471860] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1548.632171] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1549.000986] Lustre: lustre-OST0000: new disk, initializing [ 1549.006593] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1549.014596] Lustre: Skipped 1 previous similar message [ 1549.033489] Lustre: 54936:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1550.611741] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1550.617609] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1550.676485] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1555.922987] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1564.962463] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1567.167788] Lustre: DEBUG MARKER: create test files [ 1602.302893] Lustre: server umount lustre-OST0000 complete [ 1606.426165] Lustre: server umount lustre-MDT0000 complete [ 1616.859390] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 1619.713871] Key type lgssc unregistered [ 1620.015495] LNet: 56569:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1620.030760] LNetError: 56569:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1621.094694] LNet: Removed LNI 192.168.202.134@tcp [ 1622.182277] Key type .llcrypt unregistered [ 1622.191839] Key type ._llcrypt unregistered [ 1650.017375] Key type ._llcrypt registered [ 1650.019533] Key type .llcrypt registered [ 1650.118959] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 1651.457018] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1651.499712] alg: No test for adler32 (adler32-zlib) [ 1652.596126] Lustre: Lustre: Build Version: 2.17.57_1_g689e325 [ 1652.860858] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 1654.543390] Key type lgssc registered [ 1655.628331] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1662.748352] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1662.772375] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1664.250513] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1668.939584] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1676.822445] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1684.532268] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1684.845974] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1689.967212] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:98 to 0x240000400:129) [ 1690.082973] Lustre: 59306:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1690.094055] Lustre: 59306:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1690.099143] Lustre: 59306:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1690.112393] Lustre: 59306:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1690.121571] Lustre: 59306:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1690.133969] Lustre: 59306:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1691.056677] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1698.991073] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1704.648155] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1712.189493] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1720.802529] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1720.807924] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1720.840875] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1721.494552] Lustre: server umount lustre-OST0000 complete [ 1724.952257] Lustre: server umount lustre-MDT0000 complete [ 1736.006723] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 1738.625508] Key type lgssc unregistered [ 1738.874531] LNet: 60916:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1738.881172] LNetError: 60916:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1738.896965] LNet: Removed LNI 192.168.202.134@tcp [ 1739.705217] Key type .llcrypt unregistered [ 1739.707380] Key type ._llcrypt unregistered [ 1770.123182] Key type ._llcrypt registered [ 1770.126980] Key type .llcrypt registered [ 1770.277823] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 1771.544115] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1771.748671] alg: No test for adler32 (adler32-zlib) [ 1772.967913] Lustre: Lustre: Build Version: 2.17.57_1_g689e325 [ 1773.238758] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 1774.919207] Key type lgssc registered [ 1775.938523] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1785.385511] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1785.437407] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1787.167673] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1795.138719] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1803.359284] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1811.539331] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1811.963184] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1817.084576] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1817.196662] Lustre: 63760:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1817.202556] Lustre: 63760:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1817.216636] Lustre: 63760:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1817.228672] Lustre: 63760:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1817.236627] Lustre: 63760:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1817.241701] Lustre: 63760:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1818.799751] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1826.402628] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1831.676463] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1873.379766] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1873.381351] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1873.411598] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1875.493452] Lustre: server umount lustre-OST0000 complete [ 1880.313437] Lustre: server umount lustre-MDT0000 complete [ 1894.483306] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 1898.232775] Key type lgssc unregistered [ 1898.578363] LNet: 65344:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1898.585284] LNetError: 65344:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1898.602781] LNet: Removed LNI 192.168.202.134@tcp [ 1899.746812] Key type .llcrypt unregistered [ 1899.748833] Key type ._llcrypt unregistered [ 1918.725645] Key type ._llcrypt registered [ 1918.728828] Key type .llcrypt registered [ 1918.832378] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 1934.652076] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 1936.063016] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1936.204522] alg: No test for adler32 (adler32-zlib) [ 1937.335143] Lustre: Lustre: Build Version: 2.17.57_1_g689e325 [ 1937.655486] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 1939.384101] Key type lgssc registered [ 1940.403555] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1983.091862] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1983.137909] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1984.687450] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1984.766643] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1985.065248] Lustre: lustre-MDT0000: new disk, initializing [ 1985.344684] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1985.417617] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1989.641821] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2000.629293] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2000.825055] Lustre: 68496:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2000.884855] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2000.889393] Lustre: Skipped 1 previous similar message [ 2000.953904] Lustre: lustre-MDT0001: new disk, initializing [ 2001.035390] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2001.124666] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2001.132240] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2005.104608] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2016.174886] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2024.247840] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2033.601263] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2033.944428] Lustre: lustre-OST0000: new disk, initializing [ 2033.950493] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2033.958120] Lustre: 70463:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2034.076499] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2040.436285] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2040.461904] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2040.673225] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2041.307899] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2051.489181] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2056.152103] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2056.380793] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2061.555796] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2061.728955] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2066.404177] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2066.407877] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2066.423881] LustreError: Skipped 1 previous similar message [ 2066.430378] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2066.455708] Lustre: Skipped 1 previous similar message [ 2070.391950] Lustre: server umount lustre-OST0000 complete [ 2076.648190] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2076.677339] Lustre: Skipped 1 previous similar message [ 2076.681824] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2076.694194] Lustre: Skipped 2 previous similar messages [ 2081.205923] Lustre: server umount lustre-MDT0000 complete [ 2085.565217] LustreError: 68490:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786722544 with bad export cookie 11479471034890594332 [ 2085.584873] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2085.585312] LustreError: 68490:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2086.028987] Lustre: server umount lustre-MDT0001 complete [ 2097.409552] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 11:49:13 (1786722553) [ 2109.185739] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 2151.364363] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 2165.322503] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 2201.905745] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2202.164646] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2202.199684] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2202.292668] Lustre: lustre-MDT0000: new disk, initializing [ 2202.395583] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2202.433228] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2206.546671] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2216.471807] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2216.594849] Lustre: 76487:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2216.664608] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2216.672811] Lustre: Skipped 1 previous similar message [ 2216.742758] Lustre: lustre-MDT0001: new disk, initializing [ 2216.821678] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2216.862381] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2216.879730] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2220.925143] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2229.373237] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2234.260777] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2240.709725] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2241.000357] Lustre: lustre-OST0000: new disk, initializing [ 2241.009719] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2241.021668] Lustre: 78456:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2241.145582] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2247.103411] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2247.750898] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2247.755547] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2247.841949] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2259.248134] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2265.866867] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2266.205270] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2270.946075] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2271.196252] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2273.760486] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2273.764919] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2273.774477] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2273.789699] Lustre: Skipped 1 previous similar message [ 2278.884195] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2278.891619] Lustre: Skipped 2 previous similar messages [ 2279.766152] Lustre: server umount lustre-OST0000 complete [ 2284.000701] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2284.028908] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2288.609165] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2288.618835] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2290.021855] Lustre: server umount lustre-MDT0000 complete [ 2293.866831] LustreError: 76482:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786722752 with bad export cookie 11479471034890595214 [ 2293.872128] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2293.879137] LustreError: 76482:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2294.289883] Lustre: server umount lustre-MDT0001 complete [ 2303.633435] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 11:52:40 (1786722760) [ 2310.435564] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2310.963368] LustreError: 80871:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2310.988063] LustreError: 80871:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2311.047705] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2315.259706] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2316.277693] LustreError: 80872:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2316.303624] LustreError: 80872:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2320.356268] LustreError: 80871:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2323.813531] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2328.695697] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2336.546442] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2344.758647] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2353.954444] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2354.224205] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2354.231528] Lustre: Skipped 1 previous similar message [ 2361.368712] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2371.360353] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2376.179242] Lustre: 83544:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 94d6dfe6-79d6-4e3e-8a1d-24b1af7f3f45 at adminstrative request [ 2378.747131] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2378.759624] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2388.964776] Lustre: MGS: Client 94d6dfe6-79d6-4e3e-8a1d-24b1af7f3f45 (at 0@lo) reconnecting [ 2388.973654] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x9f4f4615dc0a2ef9 to 0x9f4f4615dc0a31a0 [ 2389.007242] Lustre: MGC192.168.202.134@tcp: Connection restored to 0@lo (at 0@lo) [ 2413.758315] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2414.006643] Lustre: lustre-OST0001: new disk, initializing [ 2414.016741] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2414.024388] Lustre: 83937:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2422.406267] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2422.435432] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2422.575759] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2423.149121] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2435.692924] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2443.232327] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2443.233784] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2443.250913] LustreError: Skipped 1 previous similar message [ 2443.264782] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2443.277149] Lustre: Skipped 1 previous similar message [ 2443.327185] Lustre: Skipped 4 previous similar messages [ 2445.508373] Lustre: server umount lustre-OST0001 complete [ 2448.371129] LustreError: 82690:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2448.399552] LustreError: 82690:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2450.400657] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2453.507050] LustreError: 82692:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2453.549178] LustreError: 82692:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2463.718485] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2463.721834] LustreError: 83577:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2463.734868] Lustre: Skipped 7 previous similar messages [ 2463.767505] LustreError: 83577:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2464.223263] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2464.433214] Lustre: server umount lustre-OST0000 complete [ 2472.423074] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2472.429063] LustreError: Skipped 1 previous similar message [ 2472.434273] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2472.446370] Lustre: Skipped 2 previous similar messages [ 2477.332274] Lustre: server umount lustre-MDT0000 complete [ 2481.390459] LustreError: 80853:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786722940 with bad export cookie 11479471034890596768 [ 2481.399746] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2481.403068] LustreError: 80853:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2481.718785] Lustre: server umount lustre-MDT0001 complete [ 2490.995963] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 11:55:48 (1786722948) [ 2492.491820] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2494.199119] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 11:55:51 (1786722951) [ 2495.726816] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2497.823975] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 11:55:54 (1786722954) [ 2499.806963] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2501.781539] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 11:55:58 (1786722958) [ 2503.778570] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2506.020231] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 11:56:02 (1786722962) [ 2507.551399] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2513.797664] loop: module loaded [ 2513.840141] loop0: detected capacity change from 0 to 4294967296 [ 2513.904487] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2515.644328] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2520.018549] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2525.209614] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2552.875159] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2552.900339] LustreError: 88065:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2552.906346] LustreError: 88065:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2552.912995] LustreError: 88065:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2552.922022] LustreError: 88065:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2575.527631] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 2591.018677] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 2631.857299] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2632.120858] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2632.170678] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2632.234769] Lustre: lustre-MDT0000: new disk, initializing [ 2632.334345] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2632.344278] Lustre: Skipped 1 previous similar message [ 2632.364294] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2637.598624] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2649.149146] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2649.288328] Lustre: 90950:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2649.536708] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2654.913643] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2666.760358] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2673.780135] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2681.659950] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2682.210525] Lustre: 92921:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2689.158145] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2692.157576] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2692.339206] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2699.670813] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2705.190286] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2705.603245] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2711.517765] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2711.737109] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2718.176293] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2718.181789] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2718.185917] LustreError: Skipped 1 previous similar message [ 2718.186750] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2718.186754] Lustre: Skipped 3 previous similar messages [ 2718.206429] Lustre: Skipped 2 previous similar messages [ 2720.170977] Lustre: server umount lustre-OST0000 complete [ 2730.791468] Lustre: server umount lustre-MDT0000 complete [ 2733.546720] LustreError: 90963:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2733.562021] LustreError: 90963:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2735.254878] LustreError: 90944:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786723194 with bad export cookie 11479471034890597384 [ 2735.257978] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2735.268892] LustreError: 90944:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2735.535220] Lustre: server umount lustre-MDT0001 complete [ 2745.984995] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 12:00:02 (1786723202) [ 2754.293037] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2759.995440] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2769.391853] LustreError: 95333:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2769.439430] LustreError: 95333:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2772.107171] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2772.616648] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2772.622444] Lustre: Skipped 3 previous similar messages [ 2778.844765] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2789.382547] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2799.608499] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2810.939351] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2819.244518] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2828.682192] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2834.078236] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2834.407960] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2840.314555] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2840.630431] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2848.705180] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2848.967867] Lustre: lustre-OST0001: new disk, initializing [ 2848.970919] Lustre: Skipped 2 previous similar messages [ 2848.975397] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2848.985904] Lustre: Skipped 3 previous similar messages [ 2848.991587] Lustre: 98721:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2857.348838] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2858.742073] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2858.761180] Lustre: Skipped 2 previous similar messages [ 2858.773118] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2858.901213] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2871.293384] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2879.191410] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2879.556785] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2886.344886] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2886.625681] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2910.539826] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2946.183730] Lustre: Failing over lustre-MDT0000 [ 2946.529104] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2946.548904] LustreError: Skipped 1 previous similar message [ 2946.562258] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2946.584599] Lustre: Skipped 2 previous similar messages [ 2946.592493] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2946.601584] Lustre: Skipped 4 previous similar messages [ 2946.897028] Lustre: server umount lustre-MDT0000 complete [ 2951.048312] LustreError: 98385:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.202.34@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2967.520568] Lustre: 66720:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786723410/real 1786723410] req@00000000b9b8fa9b x1873514223408256/t0(0) o400->MGC192.168.202.134@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1786723426 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2967.542046] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2974.506696] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2974.510553] LDISKFS-fs (dm-0): recovery complete [ 2974.521508] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2977.767859] LustreError: 66718:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@0000000036b735eb x1873514223417216/t0(0) o250->MGC192.168.202.134@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2978.243420] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2982.761763] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2982.765679] Lustre: 95329:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2983.399514] Lustre: 96062:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2983.419286] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2983.425166] Lustre: 96062:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 1 previous similar message [ 2983.587217] LustreError: 101576:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2983.927267] LustreError: 101576:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2984.295532] LustreError: 101576:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2984.320360] LustreError: 101576:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 2984.395715] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2984.663296] LustreError: 101576:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2984.673764] LustreError: 101576:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 2984.687188] Lustre: 101576:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2984.706214] Lustre: 101576:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 7 previous similar messages [ 2985.338405] LustreError: 101576:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2985.357830] LustreError: 101576:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 2 previous similar messages [ 2985.699251] LustreError: 101576:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2985.702794] LustreError: 101576:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 2 previous similar messages [ 2986.720686] Lustre: 101576:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2986.737836] Lustre: 101576:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 5 previous similar messages [ 2987.427238] LustreError: 101576:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2987.476513] LustreError: 101576:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 5 previous similar messages [ 2987.809748] LustreError: 101576:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2987.827635] LustreError: 101576:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 5 previous similar messages [ 2990.737480] Lustre: 101576:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2990.753145] Lustre: 101576:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 11 previous similar messages [ 2991.748596] LustreError: 101576:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2991.783474] LustreError: 101576:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 2992.127150] LustreError: 101576:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2992.149838] LustreError: 101576:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 2998.960884] Lustre: 101576:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2998.981373] Lustre: 101576:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 24 previous similar messages [ 2999.953319] LustreError: 101576:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2999.962202] LustreError: 101576:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 24 previous similar messages [ 3000.303197] LustreError: 101576:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3000.325472] LustreError: 101576:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 24 previous similar messages [ 3015.033603] Lustre: 101576:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3015.054830] Lustre: 101576:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 49 previous similar messages [ 3016.016944] LustreError: 101576:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3016.037455] LustreError: 101576:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 49 previous similar messages [ 3016.380988] LustreError: 101576:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3016.396513] LustreError: 101576:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 49 previous similar messages [ 3042.159243] Lustre: 101576:0:(ldlm_lib.c:2394:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3042.173782] Lustre: 101576:0:(ldlm_lib.c:2404:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3042.186700] Lustre: 101576:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client c721a5e5-41e6-4c31-90b3-575bba1d31d7@192.168.202.34@tcp [ 3042.208906] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3042.236498] Lustre: 101576:0:(ldlm_lib.c:1914:abort_req_replay_queue()) @@@ aborted: req@000000003098eaec x1873514206403456/t0(8589934778) o35->c721a5e5-41e6-4c31-90b3-575bba1d31d7@192.168.202.34@tcp:156/0 lens 392/0 e 0 to 0 dl 1786723511 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3042.289114] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3042.309673] Lustre: lustre-MDT0000: Denying connection for new client c721a5e5-41e6-4c31-90b3-575bba1d31d7 (at 192.168.202.34@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3042.342600] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3042.391424] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3042.400326] Lustre: Skipped 2 previous similar messages [ 3042.422471] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3042.466365] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 3042.471557] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:48 to 0x2c0000401:545) [ 3063.776428] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3063.791129] LustreError: Skipped 1 previous similar message [ 3066.811223] Lustre: server umount lustre-OST0000 complete [ 3075.558353] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 3075.571477] Lustre: Skipped 7 previous similar messages [ 3080.694461] LustreError: 101964:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3080.714778] LustreError: 101964:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 46 previous similar messages [ 3086.305122] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3086.670715] Lustre: server umount lustre-OST0001 complete [ 3093.987797] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3093.997361] Lustre: Skipped 7 previous similar messages [ 3097.669138] Lustre: server umount lustre-MDT0000 complete [ 3102.496907] LustreError: 95312:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786723561 with bad export cookie 11479471034890648603 [ 3102.504063] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3102.514883] LustreError: 95312:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3103.204456] Lustre: server umount lustre-MDT0001 complete [ 3115.299770] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 12:06:11 (1786723571) [ 3124.291720] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3124.296843] Lustre: *** cfs_fail_loc=197, val=0*** [ 3139.935825] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3145.057791] LustreError: 103556:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.134@tcp: failed processing log, type 4: rc = -110 [ 3150.204346] Lustre: *** cfs_fail_loc=197, val=0*** [ 3150.207366] Lustre: Skipped 65 previous similar messages [ 3170.783353] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3170.788294] Lustre: Skipped 3 previous similar messages [ 3179.420854] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3188.924550] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3192.496394] Lustre: server umount lustre-OST0000 complete [ 3204.196524] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 12:07:40 (1786723660) [ 3233.659979] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3241.578138] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3251.932227] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3256.254337] LustreError: 105210:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3256.276704] LustreError: 105210:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3256.290086] LustreError: 105210:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 22, retries 0, failed: rc = -5 [ 3257.023815] Lustre: server umount lustre-MDT0000 complete [ 3302.820568] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 3320.051675] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 3362.095901] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3362.392096] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3362.419607] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3362.480109] Lustre: lustre-MDT0000: new disk, initializing [ 3362.528350] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3366.737182] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3378.937892] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3379.086670] Lustre: 109424:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3379.376659] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3383.475321] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3393.795559] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3400.583864] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3409.158861] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3409.408501] Lustre: 111400:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3416.621743] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3419.331223] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3427.472788] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3432.358250] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3432.548241] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3436.812838] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3437.030522] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3440.096651] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3440.104575] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3440.109368] LustreError: Skipped 4 previous similar messages [ 3440.113951] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3440.123572] Lustre: Skipped 2 previous similar messages [ 3440.145632] Lustre: Skipped 8 previous similar messages [ 3445.257977] Lustre: server umount lustre-OST0000 complete [ 3454.988585] Lustre: server umount lustre-MDT0000 complete [ 3455.456960] LustreError: 110659:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3455.491951] LustreError: 110659:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 10 previous similar messages [ 3459.526927] LustreError: 109417:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786723918 with bad export cookie 11479471034890659278 [ 3459.538659] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3459.542435] LustreError: 109417:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3460.040463] Lustre: server umount lustre-MDT0001 complete [ 3472.630881] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 12:12:08 (1786723928) [ 3484.063406] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 3488.207041] Key type lgssc unregistered [ 3488.707348] LNet: 113891:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3488.713046] LNetError: 113891:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3488.723650] LNet: Removed LNI 192.168.202.134@tcp [ 3490.223454] Key type .llcrypt unregistered [ 3490.226447] Key type ._llcrypt unregistered [ 3509.769898] Key type ._llcrypt registered [ 3509.775162] Key type .llcrypt registered [ 3509.903559] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 3527.916978] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 3529.252848] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3529.329882] alg: No test for adler32 (adler32-zlib) [ 3530.723463] Lustre: Lustre: Build Version: 2.17.57_1_g689e325 [ 3530.995986] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 3532.711292] Key type lgssc registered [ 3534.154904] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3591.849593] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3591.880512] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3593.285970] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3593.337197] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3593.439729] Lustre: lustre-MDT0000: new disk, initializing [ 3593.554378] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3593.570586] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3598.791195] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3611.889203] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3612.266095] Lustre: lustre-OST0000: new disk, initializing [ 3612.276526] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3612.282884] Lustre: Skipped 1 previous similar message [ 3612.296345] Lustre: 118070:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3612.471420] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3619.981810] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3619.994571] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3620.038585] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3620.051267] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3631.448717] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3631.605467] Lustre: lustre-OST0001: new disk, initializing [ 3631.609187] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3631.616645] Lustre: 118932:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3631.684485] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3639.306383] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3639.463496] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3639.475227] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3639.505454] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3654.651138] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3654.657493] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3654.684893] Lustre: Skipped 1 previous similar message [ 3654.721649] Lustre: Skipped 1 previous similar message [ 3656.893511] Lustre: server umount lustre-MDT0000 complete [ 3665.238238] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3672.408468] LustreError: 117255:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786724131 with bad export cookie 2847428698167132138 [ 3672.416802] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3672.429739] LustreError: 117255:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3672.611528] Lustre: server umount lustre-OST0000 complete [ 3678.704210] Lustre: server umount lustre-OST0001 complete [ 3691.530640] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 12:15:47 (1786724147) [ 3726.481138] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 12:16:22 (1786724182) [ 3742.845802] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 3779.358853] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 12:17:15 (1786724235) [ 3829.780275] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 3845.698187] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 3894.007419] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3894.234990] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3894.260757] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3894.334286] Lustre: lustre-MDT0000: new disk, initializing [ 3894.405450] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3894.432581] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3900.147272] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3913.250113] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3913.423211] Lustre: 126734:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3913.529882] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3913.541823] Lustre: Skipped 1 previous similar message [ 3913.641159] Lustre: lustre-MDT0001: new disk, initializing [ 3913.776404] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3913.805055] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3913.821853] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3920.351851] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3932.305278] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3939.728569] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3947.731855] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3948.083942] Lustre: lustre-OST0000: new disk, initializing [ 3948.094551] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3948.106074] Lustre: 128705:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3948.274143] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3954.748463] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3954.757938] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3954.841057] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3955.564308] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3966.904459] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3972.781378] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3973.027841] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3977.497247] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3977.655266] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3980.773226] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3980.773849] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3980.817787] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3980.828806] Lustre: Skipped 1 previous similar message [ 3985.892700] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3985.906046] Lustre: Skipped 1 previous similar message [ 3991.010116] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3991.027648] Lustre: Skipped 1 previous similar message [ 3994.080242] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3994.281587] Lustre: server umount lustre-OST0000 complete [ 4000.736413] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4000.744808] LustreError: Skipped 1 previous similar message [ 4000.747974] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4000.754040] Lustre: Skipped 1 previous similar message [ 4000.762623] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4004.613758] Lustre: server umount lustre-MDT0000 complete [ 4008.736037] LustreError: 126727:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786724467 with bad export cookie 2847428698167133363 [ 4008.742161] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4008.748509] LustreError: 126727:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4009.155371] Lustre: server umount lustre-MDT0001 complete [ 4018.223165] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 12:21:14 (1786724474) [ 4025.051988] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4025.376492] LustreError: 131119:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4025.402653] LustreError: 131119:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 4025.460312] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4029.837453] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4030.970346] LustreError: 131120:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4036.074787] LustreError: 131119:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4038.952298] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4045.620721] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4055.847576] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4063.942597] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4071.444241] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4071.860536] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4071.868784] Lustre: Skipped 1 previous similar message [ 4078.741925] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4086.656840] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4102.303035] Lustre: *** cfs_fail_loc=159, val=0*** [ 4117.957735] Lustre: lustre-MDT0000: Client 8bf0a5ec-0843-4b74-a303-9cb7033c040a (at 192.168.202.34@tcp) reconnecting [ 4119.499628] Lustre: *** cfs_fail_loc=159, val=0*** [ 4119.503921] Lustre: Skipped 8 previous similar messages [ 4134.803578] Lustre: lustre-MDT0000: Client 8bf0a5ec-0843-4b74-a303-9cb7033c040a (at 192.168.202.34@tcp) reconnecting [ 4155.875396] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4155.884519] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4155.898147] Lustre: Skipped 1 previous similar message [ 4155.902994] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4155.915822] Lustre: Skipped 2 previous similar messages [ 4159.286359] Lustre: server umount lustre-OST0000 complete [ 4167.137548] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4167.137980] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4167.155259] LustreError: Skipped 1 previous similar message [ 4167.181143] Lustre: Skipped 2 previous similar messages [ 4172.256366] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4172.262399] Lustre: Skipped 4 previous similar messages [ 4177.375877] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4177.632989] Lustre: server umount lustre-MDT0000 complete [ 4181.482870] LustreError: 131119:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4182.275023] LustreError: 131100:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786724641 with bad export cookie 2847428698167134245 [ 4182.292350] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4183.176647] Lustre: server umount lustre-MDT0001 complete [ 4198.604661] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 4202.362406] Key type lgssc unregistered [ 4202.902672] LNet: 134933:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4202.913705] LNetError: 134933:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4202.947641] LNet: Removed LNI 192.168.202.134@tcp [ 4204.380618] Key type .llcrypt unregistered [ 4204.382192] Key type ._llcrypt unregistered [ 4235.572945] Key type ._llcrypt registered [ 4235.575318] Key type .llcrypt registered [ 4235.771353] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 4237.057717] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4237.140346] alg: No test for adler32 (adler32-zlib) [ 4238.467529] Lustre: Lustre: Build Version: 2.17.57_1_g689e325 [ 4238.708068] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 4240.447947] Key type lgssc registered [ 4241.636351] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4249.018763] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4249.060747] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4250.459615] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4255.459566] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4260.836189] LustreError: 136349:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4260.859364] LustreError: 136349:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4264.183573] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4264.554192] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4269.293843] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4277.275188] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4283.620244] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4291.594836] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4291.919050] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4296.077724] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4299.726897] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4311.584512] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4322.734160] Lustre: *** cfs_fail_loc=159, val=0*** [ 4339.100764] Lustre: lustre-MDT0000: Client de467d33-ec84-4c04-9f0d-1fa9a57df1f8 (at 192.168.202.34@tcp) reconnecting [ 4341.623528] Lustre: *** cfs_fail_loc=159, val=0*** [ 4357.080729] Lustre: lustre-MDT0001: Client de467d33-ec84-4c04-9f0d-1fa9a57df1f8 (at 192.168.202.34@tcp) reconnecting [ 4359.285356] Lustre: *** cfs_fail_loc=159, val=0*** [ 4359.290373] Lustre: Skipped 3 previous similar messages [ 4375.099159] Lustre: lustre-MDT0001: Client de467d33-ec84-4c04-9f0d-1fa9a57df1f8 (at 192.168.202.34@tcp) reconnecting [ 4379.666335] Lustre: 139390:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4385.145867] Lustre: *** cfs_fail_loc=159, val=0*** [ 4385.158018] Lustre: Skipped 4 previous similar messages [ 4401.579301] Lustre: lustre-MDT0000: Client 164ec8e7-d3d9-48cf-ba30-52740143ad46 (at 192.168.202.34@tcp) reconnecting [ 4405.101665] Lustre: *** cfs_fail_loc=159, val=0*** [ 4405.108141] Lustre: Skipped 14 previous similar messages [ 4421.525071] Lustre: lustre-MDT0000: Client 164ec8e7-d3d9-48cf-ba30-52740143ad46 (at 192.168.202.34@tcp) reconnecting [ 4423.485851] Lustre: 139682:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4432.360755] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4432.376950] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4432.399856] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4432.426293] Lustre: Skipped 1 previous similar message [ 4436.384233] Lustre: server umount lustre-OST0000 complete [ 4442.271205] Lustre: server umount lustre-MDT0000 complete [ 4442.596456] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4442.608269] LustreError: 136345:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4442.611693] Lustre: Skipped 1 previous similar message [ 4442.643543] LustreError: 136345:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4447.739498] LustreError: 139566:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4447.772157] LustreError: 139566:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4448.056253] LustreError: 139632:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786724906 with bad export cookie 14743703481517336085 [ 4448.080037] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4448.082908] LustreError: 139632:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4449.440873] Lustre: server umount lustre-MDT0001 complete [ 4464.388883] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 4466.826904] Key type lgssc unregistered [ 4467.114879] LNet: 140737:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4467.128684] LNetError: 140737:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4467.146473] LNet: Removed LNI 192.168.202.134@tcp [ 4468.157493] Key type .llcrypt unregistered [ 4468.161131] Key type ._llcrypt unregistered [ 4498.249649] Key type ._llcrypt registered [ 4498.252881] Key type .llcrypt registered [ 4498.338543] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 4499.744537] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4499.786230] alg: No test for adler32 (adler32-zlib) [ 4500.850682] Lustre: Lustre: Build Version: 2.17.57_1_g689e325 [ 4501.060601] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 4502.688407] Key type lgssc registered [ 4504.024652] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4511.847322] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4511.870239] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4513.452261] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4518.408846] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4524.052589] LustreError: 142151:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4524.082623] LustreError: 142151:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4527.351865] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4527.808278] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4533.071748] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4543.244508] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4550.832880] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4558.153120] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4558.754238] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4559.825109] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4566.980546] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4573.139036] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4577.457765] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4584.808991] Lustre: 144737:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4587.243668] Lustre: 144786:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4588.813836] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4589.944088] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4595.210396] Lustre: 144932:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4596.444848] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4597.514267] Lustre: 145040:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4603.874663] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4603.874724] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4603.876349] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4603.917330] Lustre: Skipped 1 previous similar message [ 4608.797485] Lustre: server umount lustre-OST0000 complete [ 4614.112682] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4614.138813] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4614.147977] Lustre: Skipped 2 previous similar messages [ 4619.236465] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4619.259190] Lustre: Skipped 1 previous similar message [ 4619.451126] Lustre: server umount lustre-MDT0000 complete [ 4623.328637] LustreError: 142146:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4623.367740] LustreError: 142146:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4624.686488] LustreError: 142132:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786725083 with bad export cookie 1399995971177665473 [ 4624.686931] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4624.692973] LustreError: 142132:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4625.158570] Lustre: server umount lustre-MDT0001 complete [ 4642.299312] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 4647.081643] Key type lgssc unregistered [ 4647.504488] LNet: 146096:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4647.512907] LNetError: 146096:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4647.535159] LNet: Removed LNI 192.168.202.134@tcp [ 4648.766625] Key type .llcrypt unregistered [ 4648.773957] Key type ._llcrypt unregistered [ 4685.794542] Key type ._llcrypt registered [ 4685.796821] Key type .llcrypt registered [ 4686.005339] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 4687.605436] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4687.643172] alg: No test for adler32 (adler32-zlib) [ 4689.098873] Lustre: Lustre: Build Version: 2.17.57_1_g689e325 [ 4689.547940] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 4691.335188] Key type lgssc registered [ 4693.051573] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4704.369486] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4704.425743] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4706.300830] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4711.567766] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4716.514130] LustreError: 147516:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4716.527346] LustreError: 147516:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4720.001441] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4720.287641] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4725.108370] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4734.241615] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4740.521808] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4752.666275] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4753.515717] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4764.819800] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4765.866296] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4775.251851] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4777.184871] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4785.308945] Lustre: *** cfs_fail_loc=159, val=0*** [ 4800.921769] Lustre: lustre-MDT0000: Client a434fa82-afdd-443e-ad3c-8dee9177994d (at 192.168.202.34@tcp) reconnecting [ 4807.140538] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4807.153269] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4807.165349] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4808.176384] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4808.186868] Lustre: Skipped 2 previous similar messages [ 4812.851668] Lustre: server umount lustre-OST0000 complete [ 4817.377212] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4817.386641] LustreError: Skipped 6 previous similar messages [ 4817.393465] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4817.411508] Lustre: Skipped 1 previous similar message [ 4817.419791] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4818.412789] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4822.944478] Lustre: server umount lustre-MDT0000 complete [ 4823.521174] LustreError: 149295:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4823.534306] LustreError: 149295:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4827.521035] LustreError: 147497:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786725286 with bad export cookie 4978123022788615809 [ 4827.525871] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4827.541355] LustreError: 147497:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4828.228904] Lustre: server umount lustre-MDT0001 complete [ 4841.218255] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 4844.400505] Key type lgssc unregistered [ 4844.741729] LNet: 151187:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4844.752511] LNetError: 151187:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4844.769320] LNet: Removed LNI 192.168.202.134@tcp [ 4845.835460] Key type .llcrypt unregistered [ 4845.838261] Key type ._llcrypt unregistered [ 4877.950124] Key type ._llcrypt registered [ 4877.952224] Key type .llcrypt registered [ 4878.096815] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 4879.036198] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4879.063582] alg: No test for adler32 (adler32-zlib) [ 4880.338944] Lustre: Lustre: Build Version: 2.17.57_1_g689e325 [ 4880.605071] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 4882.353564] Key type lgssc registered [ 4883.921812] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4892.824733] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4892.859718] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4894.458476] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4900.047706] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4904.947549] LustreError: 152601:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4904.971219] LustreError: 152601:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4909.032752] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4909.465276] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4914.927779] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4925.845756] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4934.525978] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4942.826203] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4943.284558] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4949.816758] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4955.577386] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4958.592899] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4964.797978] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4984.043098] Lustre: lustre-OST0000: evicting 67ed5d54-8cb7-4977-9c03-01a876e11002 (at 192.168.202.34@tcp) by administrative request [ 4984.053802] Lustre: Skipped 1 previous similar message [ 5016.041166] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5016.047310] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5016.051704] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5016.055679] Lustre: Skipped 1 previous similar message [ 5016.073919] Lustre: Skipped 1 previous similar message [ 5021.161632] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5021.175270] Lustre: Skipped 1 previous similar message [ 5026.273219] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5026.289579] Lustre: Skipped 1 previous similar message [ 5027.295179] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5027.709939] Lustre: server umount lustre-OST0000 complete [ 5036.513104] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5036.517247] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5036.535123] Lustre: Skipped 1 previous similar message [ 5036.553372] Lustre: Skipped 1 previous similar message [ 5038.752037] Lustre: server umount lustre-MDT0000 complete [ 5041.640115] LustreError: 152601:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5041.662882] LustreError: 152601:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5043.097477] LustreError: 152582:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786725501 with bad export cookie 5687470149475110296 [ 5043.101824] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5043.117830] LustreError: 152582:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5043.660453] Lustre: server umount lustre-MDT0001 complete [ 5055.598489] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 5058.468834] Key type lgssc unregistered [ 5058.767868] LNet: 156679:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5058.777397] LNetError: 156679:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5058.806779] LNet: Removed LNI 192.168.202.134@tcp [ 5059.832490] Key type .llcrypt unregistered [ 5059.835322] Key type ._llcrypt unregistered [ 5149.140688] Key type ._llcrypt registered [ 5149.142987] Key type .llcrypt registered [ 5149.277581] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 5150.443174] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5150.500455] alg: No test for adler32 (adler32-zlib) [ 5151.703780] Lustre: Lustre: Build Version: 2.17.57_1_g689e325 [ 5151.884258] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 5153.575534] Key type lgssc registered [ 5154.722611] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5163.307465] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5163.346761] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5165.074741] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5170.223682] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5174.241737] LustreError: 159185:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5174.253333] LustreError: 159185:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5179.361877] LustreError: 159186:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5180.000842] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5180.420682] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5185.598416] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5194.582408] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5202.760877] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5211.464462] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5212.055226] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5219.111335] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5225.401750] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5227.651593] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5234.614900] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 5250.022550] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5250.043744] Lustre: Skipped 1 previous similar message [ 5250.047332] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5253.495616] Lustre: server umount lustre-OST0000 complete [ 5260.261489] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5260.271774] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5260.301148] Lustre: Skipped 1 previous similar message [ 5260.312766] Lustre: Skipped 2 previous similar messages [ 5264.458197] Lustre: server umount lustre-MDT0000 complete [ 5265.378824] LustreError: 159182:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5265.397165] LustreError: 159182:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5268.807424] LustreError: 159166:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786725727 with bad export cookie 1111791803835082185 [ 5268.813257] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5268.814155] LustreError: 159166:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5269.323944] Lustre: server umount lustre-MDT0001 complete [ 5284.481449] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 5287.115055] Key type lgssc unregistered [ 5287.475536] LNet: 162756:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5287.484491] LNetError: 162756:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5287.500243] LNet: Removed LNI 192.168.202.134@tcp [ 5288.394350] Key type .llcrypt unregistered [ 5288.399753] Key type ._llcrypt unregistered [ 5319.826739] Key type ._llcrypt registered [ 5319.836156] Key type .llcrypt registered [ 5320.026656] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 5321.254306] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5321.415541] alg: No test for adler32 (adler32-zlib) [ 5322.533598] Lustre: Lustre: Build Version: 2.17.57_1_g689e325 [ 5322.806670] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 5324.543213] Key type lgssc registered [ 5325.650599] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5343.276976] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 12:43:20 (1786725800) [ 5360.540681] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 5376.330490] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 5434.102061] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5434.142834] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5435.354871] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5435.373656] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5435.451775] Lustre: lustre-MDT0000: new disk, initializing [ 5435.521851] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5435.537139] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5440.620404] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5453.398857] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5453.495973] Lustre: 167047:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5453.537489] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5453.540307] Lustre: Skipped 1 previous similar message [ 5453.592948] Lustre: lustre-MDT0001: new disk, initializing [ 5453.706253] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5453.747258] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5453.757566] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5459.062570] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5472.183921] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5481.306551] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5489.867069] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5490.142790] Lustre: lustre-OST0000: new disk, initializing [ 5490.146630] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5490.154794] Lustre: 169016:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5490.317661] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5497.571764] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5500.541174] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5500.566796] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5500.743868] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5509.203897] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5521.377313] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5521.383212] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5521.390783] LustreError: Skipped 1 previous similar message [ 5521.391815] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5521.418111] Lustre: Skipped 1 previous similar message [ 5526.497454] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5526.513872] Lustre: Skipped 2 previous similar messages [ 5526.747301] Lustre: server umount lustre-OST0000 complete [ 5530.592448] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5530.602357] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5530.618472] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5531.618213] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5536.505193] Lustre: server umount lustre-MDT0000 complete [ 5541.213089] LustreError: 167037:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786726000 with bad export cookie 16539710220014009068 [ 5541.236359] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5541.584224] Lustre: server umount lustre-MDT0001 complete [ 5556.969200] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 5560.184476] Key type lgssc unregistered [ 5560.451703] LNet: 170923:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5560.456926] LNetError: 170923:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5561.509527] LNet: Removed LNI 192.168.202.134@tcp [ 5562.216337] Key type .llcrypt unregistered [ 5562.218352] Key type ._llcrypt unregistered [ 5590.428770] Key type ._llcrypt registered [ 5590.431414] Key type .llcrypt registered [ 5590.555690] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 5592.033081] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5592.116600] alg: No test for adler32 (adler32-zlib) [ 5593.277779] Lustre: Lustre: Build Version: 2.17.57_1_g689e325 [ 5593.568759] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 5595.287516] Key type lgssc registered [ 5596.435902] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5605.399803] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5605.427599] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5606.959105] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5611.246381] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5617.127146] LustreError: 172341:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5617.156067] LustreError: 172341:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5621.336386] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5621.784956] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5626.399995] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5634.588262] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5640.782564] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5649.146738] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5649.374351] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5652.471849] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5655.740475] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5663.791748] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5674.043331] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5674.070643] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5674.084730] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5674.093255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.110427] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5675.515429] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5675.543441] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5675.560071] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5675.570966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.591693] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5676.775933] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5676.795975] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5676.811297] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5676.819707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.831019] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5677.794707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.785159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.890259] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5679.899837] Lustre: Skipped 2 previous similar messages [ 5679.931037] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5679.946687] Lustre: Skipped 2 previous similar messages [ 5679.952833] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5679.959925] Lustre: Skipped 2 previous similar messages [ 5679.971188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.986548] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5679.989546] Lustre: Skipped 2 previous similar messages [ 5680.989811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.110071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.104042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.039545] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5684.053182] Lustre: Skipped 3 previous similar messages [ 5684.075687] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5684.084768] Lustre: Skipped 3 previous similar messages [ 5684.089621] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5684.093759] Lustre: Skipped 3 previous similar messages [ 5684.097282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.106535] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5684.117212] Lustre: Skipped 3 previous similar messages [ 5685.224186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.168485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.218290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.280434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.410101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.524513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.845126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.795902] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5692.800737] Lustre: Skipped 7 previous similar messages [ 5692.821843] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5692.834839] Lustre: Skipped 7 previous similar messages [ 5692.843643] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5692.848658] Lustre: Skipped 7 previous similar messages [ 5692.855076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.883461] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5692.886491] Lustre: Skipped 7 previous similar messages [ 5694.128386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.289820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.445079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.688663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.693872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.821245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.786812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.857629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.902199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.076083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.920186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.739678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.830432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.679643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.667661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.580751] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5709.592984] Lustre: Skipped 15 previous similar messages [ 5709.610953] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5709.618590] Lustre: Skipped 15 previous similar messages [ 5709.622063] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5709.626371] Lustre: Skipped 15 previous similar messages [ 5709.629119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.640021] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5709.647442] Lustre: Skipped 15 previous similar messages [ 5710.737453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.787132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.952082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.139996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.266666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.413949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.468251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.671398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.596124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.759105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.903698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.267276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.694022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.780620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.894369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.220829] LustreError: 172360:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5728.233198] LustreError: 172360:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5728.252781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.857972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.254266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.279159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.431685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.207108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.242335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.234878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.316753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.394026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.346198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.508990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.462285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.556291] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5742.561061] Lustre: Skipped 28 previous similar messages [ 5742.580893] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5742.601642] Lustre: Skipped 28 previous similar messages [ 5742.609604] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5742.615028] Lustre: Skipped 28 previous similar messages [ 5742.623352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.652102] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5742.663122] Lustre: Skipped 28 previous similar messages [ 5743.507070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.327681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.165583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.172148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.092598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.183345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.222913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.305082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.215695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.262186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.180789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.221624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.107576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.960636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.739223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.811359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.926664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.788141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.715656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.585744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.509701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.389653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.311093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.259716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.172212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.175231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.107211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.131163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.223625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.152328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.210256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.191440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.231356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.162263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.154096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.097954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.114133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.083420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.825589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.662940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.057521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.369903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.505768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.568249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.444891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.525904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.482577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.638320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.855246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.857177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.765655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.943599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.803266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.749922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.698865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.669293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.681067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.925816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.847890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.848473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.692771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.582959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.842135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.308126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.226451] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5807.232452] Lustre: Skipped 64 previous similar messages [ 5807.242422] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5807.259063] Lustre: Skipped 64 previous similar messages [ 5807.266581] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5807.277589] Lustre: Skipped 64 previous similar messages [ 5807.286321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.310947] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5807.313822] Lustre: Skipped 64 previous similar messages [ 5808.383478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.444413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.457605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.435579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.350548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.276147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.931528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.291683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.831523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.210567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.539419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.841866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.195285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.684770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.221852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.462060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.810185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.016196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.199485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.672225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.952314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.788599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.093622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.101333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.393075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.651092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.648447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.995371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.270499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.184084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.531365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.076745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.306039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.617580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.485746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.961211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.860188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.774365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.744244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.294720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.323067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.152720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.003797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.174758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.299176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.272083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.429882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.272667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.366879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.263561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.499887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.556077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.515200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.554113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.430664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.375186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.261232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.234797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.000907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.998705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.915486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.012183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.022339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.334476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.556478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.798273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.028700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.237678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.088221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.102149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.148667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.064068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.211498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.388986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.630353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.939385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.929093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.346226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.679541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.231637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.203213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.435688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.764971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.028949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.191555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.553798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.509743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.494563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.569853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.548716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.757060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.836288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.183409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.212240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.294978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.055561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.136244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.003705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.148673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.134877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.184881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.234348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.480642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.360224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.546372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.264925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.108683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.949746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.973320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.834642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.897126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.108780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.355618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.352572] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5935.355982] Lustre: Skipped 113 previous similar messages [ 5935.377717] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5935.386383] Lustre: Skipped 113 previous similar messages [ 5935.390586] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5935.394644] Lustre: Skipped 113 previous similar messages [ 5935.400191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.411155] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5935.413705] Lustre: Skipped 113 previous similar messages [ 5936.127312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.046992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.101536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.026185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.183602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.200708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.340577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.406440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.405838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.366997] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.360872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.205058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.205518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.066365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.031199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.831204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.775649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.534600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.575635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.370181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.370089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.223184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.242132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.146891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.040151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.763862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.743088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.497835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.525471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.258051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.250216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.287237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.274035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.276099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.281744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.143525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.980879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.058418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.117600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.370950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.411257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.369550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.381511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.584810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.662821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.554788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.469559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.021042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.428447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.682185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.072390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.505920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.675759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.840968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.975887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.224520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.074254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.275836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.667220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.826047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.912616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.023579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.322076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.650062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.570574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.787194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.907073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.890131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.928574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.741466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.998378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.835262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.772594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.666342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.538415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.340839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.282202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.254974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.159497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.170057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.080137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.183496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.056352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.869847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.764452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.716366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.735857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.579562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.675422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.647551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.420667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.353892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.220496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.133564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.838507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.754517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.778198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.389034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.186168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.159379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.892564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.875730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.022356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.975471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.927699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.719637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.628197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.567043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.485251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.364244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.237138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.080880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.241611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.178678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.965572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.609465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.490951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.358683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.354049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.283173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.142621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.809577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.761067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.665660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.828215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.322648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.905890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.652569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.257342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.530654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.498955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.656875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.680569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.642847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.981662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.460511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.041566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.525160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.892858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.290659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.954811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.977707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.857657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.732956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.506127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.699339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.544498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.187168] LustreError: 172360:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6089.204279] LustreError: 172360:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6089.228552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.055928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.834529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.831182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.835466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.758622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.757362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.583494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.901059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.785818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.806786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.638373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.570190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.552849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.742352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.002964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.160122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.290566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.424638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.711051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.983298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.393445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.768465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.907311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.442945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.988745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.034285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.373189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.800459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.943453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.864913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.934838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.885413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.188221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.243733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.176842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.183191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.456693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.384778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.424331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.310507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.222489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.009116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.048847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.283118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.546929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.869645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.993422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.873286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.087661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.393295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.666506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.753899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.236671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.535642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.753715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.106534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.513028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.748981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.006192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.067240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.053510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.036627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.089594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.900651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.872733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.796935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.840810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.784869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.080099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.395065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.279373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.248344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.171489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.296898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.297543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.310590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.544956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.757415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.585303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.730596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.777479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.762087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.159937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.334092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.113763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.155357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.595775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.063289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.503501] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6192.521816] Lustre: Skipped 236 previous similar messages [ 6192.539971] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6192.554570] Lustre: Skipped 236 previous similar messages [ 6192.559352] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6192.562362] Lustre: Skipped 236 previous similar messages [ 6192.571567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.582687] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6192.585583] Lustre: Skipped 236 previous similar messages [ 6193.989147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.231510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.358231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.351818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.405444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.227048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.021684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.606245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.876315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.761563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.785554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.829235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.875710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.870250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.492368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.497957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.599025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.824856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.803907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.874619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.917294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.723945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.669340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.345663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.049874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.382984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.589159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.731387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.995937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.074617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.079797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.037363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.043963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.914066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.808720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.592411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.447142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.306707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.100491] LustreError: 172360:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6231.114824] LustreError: 172360:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6231.119316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.984441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.986315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.741616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.562160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.409543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.203112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.812475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.526623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.396755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.085261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.935317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.928793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.127661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.865905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.841250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.060572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.147926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.039487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.006467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.943537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.270768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.585709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.746557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.782553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.785749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.747464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.298232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.660215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.809332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.167441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.651659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.962303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.010572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.319104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.513623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.908952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.066960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.642823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.635656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.178643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.555208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.816784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.270660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.759467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.786181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.857334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.224547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.422664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.272912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.196295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.114742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.122084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.109162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.252907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.154536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.295927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.397298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.551939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.551431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.608139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.036889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.120330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.029323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.158327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.310389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.397940] LustreError: 172360:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6303.409946] LustreError: 172360:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6303.414847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.278841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.289990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.074699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.995651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.165049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.192385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.315640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.194834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.162202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.112301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.004294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.808294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.714725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.499542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.399472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.197077] LustreError: 172360:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6318.211104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.213069] LustreError: 172360:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6319.038854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.022805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.829357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.982790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.207183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.221846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.425084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.345379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.593331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.586028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.711693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.617124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6331.624890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.565460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.519539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.474692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.468921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.519941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.633194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6338.719750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.748241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.949113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.952967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6342.767729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6343.684142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.536955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.368864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.323851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.183920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6348.125710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6348.959152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.835408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.700696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6351.682769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.469674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.396612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6354.411420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.432700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.451647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.415697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.306531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.352473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.235906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.982256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6361.798686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6362.618551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.560541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.303370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.203065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.025654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.970327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.018619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.842119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.720432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.490540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.267927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.219958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6373.056374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.058727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.129761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.330917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6377.438922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.776220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.866185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.989624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6382.056224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.011061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6384.303194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6385.178243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6386.297682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.405798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6388.274732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6389.302410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6390.220722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.343306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6392.639961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.590802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6394.846047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6395.822280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6396.838214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.676363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.784669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.255472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6401.541481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.559641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6403.827789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6404.985980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.926876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6406.818537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6407.850443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6408.840233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6409.588856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.717653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.864449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6412.755889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.849591] LustreError: 172360:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6413.862323] LustreError: 172360:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6413.873125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6414.899068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.912157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.917106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6418.072963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.594548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.758843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6421.848501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6422.978511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.086734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.144062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.995062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6426.835903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6427.744524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6428.887594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6429.846071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.805502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.699483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6432.829357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6433.695917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6434.806438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.767561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6436.925662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.230988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6439.367227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6440.333197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.416680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.290566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6443.523538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6444.431081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.450444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6446.343720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6447.853580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6448.788433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6449.792274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6450.895553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6452.266738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6453.399404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6454.152890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.081102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6456.319269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6457.675220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6458.511308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6459.388641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6460.423686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6461.584714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6462.600427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6463.610547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6464.925884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6466.019170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6467.106878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6468.262817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6469.249788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6470.349827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6471.132816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6472.368197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6473.433940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6474.292245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6475.022938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6475.765802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6476.694836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6477.684476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6478.622802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6479.423879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6480.456975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6481.366930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6482.485128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6483.310419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6484.010375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6484.968663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6485.781121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6486.618612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6487.658623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6488.570557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6489.532459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6490.478339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6491.605952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6492.808327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6493.791166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6494.932429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6495.891502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6496.980525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6497.991307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6498.961474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6499.795853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6500.907974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6501.800464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6502.806412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6503.725717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6504.618756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6505.546744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6506.421422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6507.359737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6508.465529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6509.448622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6510.362825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6511.032341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6511.901907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6512.619616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6513.566438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6514.464510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6515.391848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6516.010323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6516.860392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6517.566184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6518.671992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6519.469846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6520.388887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6521.269223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6522.135496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6522.939230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6523.926650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6524.747062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6525.681574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6526.655128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6527.824160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6528.701772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6529.532456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6530.443804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6531.321528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6532.234690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6533.089219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6533.797210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6534.613426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6535.686070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6536.599036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6537.736197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6538.513026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6539.554140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6540.459367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6541.382463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6542.323080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6543.212617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6544.048469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6544.967140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6545.808609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6546.747585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6547.601878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6548.476355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6549.327428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6550.158700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6551.159556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6552.051530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6552.907727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6553.801259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6554.656728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6555.847260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6556.776404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6557.755288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6558.936653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6560.175828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6561.241668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6562.456219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6563.528061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6564.730235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6565.910567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6566.997504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6567.963859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6568.792468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6569.533056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6570.569296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6571.351595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6572.193113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6573.005916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6573.913967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6574.715094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6575.469531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6576.293886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6577.042873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6577.979764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6578.734836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6579.718775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6580.670762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6581.463495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6582.331134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6583.201843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6584.081748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6585.082754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6586.035518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6586.943927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6587.834922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6588.815483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6589.732530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6591.030091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6591.960684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6593.191224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6594.288109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6595.324534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6596.285992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6597.396264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6598.512293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6599.835358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6600.723777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6601.771464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6602.688541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6603.806933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6604.887979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6605.820297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6606.800476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6607.816649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6608.694642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6609.837936] LustreError: 172360:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6609.847289] LustreError: 172360:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6609.859090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6610.839230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6612.042196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6612.928587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6613.830714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6614.647565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6615.445683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6616.118255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6617.094634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6618.046818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6618.996249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6619.919625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6620.711694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6621.469818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6622.457769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6623.285896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6624.252793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6624.980740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6626.089731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6626.983677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6627.969907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6628.842441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6629.769897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6630.777069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6631.828792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6632.625971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6633.634220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6634.477722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6635.534594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6636.769899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6638.064585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6638.990515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6640.032726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6640.903682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6642.091391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6642.909592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6643.731846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6644.586726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6645.480294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6646.360501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6647.254082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6648.057780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6648.958505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6649.780464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6650.588961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6651.447267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6652.366108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6653.225173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6654.038559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6654.967808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6655.861941] LustreError: 172360:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6655.873620] LustreError: 172360:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6655.884115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6656.888485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6657.819403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6658.747917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6659.688634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6660.658227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6661.622910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6662.398193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6663.342468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6664.193602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6665.022128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6665.792495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6666.673905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6667.367817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6668.214603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6669.021966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6669.886539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6670.583101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6671.480851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6672.446083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6673.267556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6674.168667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6674.975567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6675.918821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6676.797425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6677.548385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6678.534115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6679.541964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6680.443193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6681.127392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6681.985576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6682.845166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6683.656815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6684.444260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6685.346466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6686.275314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6687.270068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6688.505136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6689.484275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6690.485455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6691.181825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6692.018503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6692.770035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6693.577710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6694.415119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6695.373905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6696.270445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6697.400851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6698.070685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6698.908026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6699.720030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6700.640942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6701.797895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6702.614946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7097.867676] LustreError: 174165:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 7097.875869] LustreError: 172360:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7789.285870] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 7789.287021] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7789.294718] LustreError: Skipped 1 previous similar message [ 7789.308634] Lustre: Skipped 521 previous similar messages [ 7789.319623] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7791.587554] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7791.590703] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7791.596540] Lustre: Skipped 1 previous similar message [ 7791.607935] LustreError: Skipped 2 previous similar messages [ 7795.612465] Lustre: server umount lustre-OST0000 complete [ 7801.827535] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7801.846034] Lustre: Skipped 1 previous similar message [ 7805.766975] Lustre: server umount lustre-MDT0000 complete [ 7806.947256] LustreError: 172340:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7806.974834] LustreError: 172340:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7808.938547] LustreError: 172320:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786728267 with bad export cookie 5668285777970776723 [ 7808.943679] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7808.950180] LustreError: 172320:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7809.182945] Lustre: server umount lustre-MDT0001 complete [ 7820.152769] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 7822.955553] Key type lgssc unregistered [ 7823.225664] LNet: 227624:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7823.234899] LNetError: 227624:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7823.251790] LNet: Removed LNI 192.168.202.134@tcp [ 7823.966583] Key type .llcrypt unregistered [ 7823.970333] Key type ._llcrypt unregistered [ 7851.115978] Key type ._llcrypt registered [ 7851.119860] Key type .llcrypt registered [ 7851.233320] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 7852.135563] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7852.158581] alg: No test for adler32 (adler32-zlib) [ 7853.382621] Lustre: Lustre: Build Version: 2.17.57_1_g689e325 [ 7853.615424] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 7855.279182] Key type lgssc registered [ 7856.561787] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7864.201690] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7864.235022] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7865.767602] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7870.696349] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 7876.066547] LustreError: 229042:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7876.089569] LustreError: 229042:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7880.532347] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7881.186943] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 7881.402614] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7886.250437] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 7897.804386] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7903.765799] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7912.317831] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7912.771968] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7919.562230] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 7927.456094] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7935.376231] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7942.656917] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7944.746565] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7944.764609] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7944.781167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7944.805660] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7944.805899] LustreError: 229062:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7947.746345] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7947.770100] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7947.798688] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7950.826520] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7950.826738] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7950.839648] Lustre: Skipped 1 previous similar message [ 7953.069465] Lustre: server umount lustre-OST0000 complete [ 7964.863387] LustreError: 229062:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7987.239115] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7987.529698] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7990.567156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7990.602068] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7994.075621] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 8000.886593] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8000.899824] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 8002.789756] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8004.626290] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 8012.116962] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9303dfcf2000.ost_server_uuid 50 [ 8013.944669] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9303dfcf2000.ost_server_uuid in IDLE state after 0 sec [ 8019.428847] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8019.436577] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8019.490107] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8021.483511] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8024.783850] Lustre: server umount lustre-OST0000 complete [ 8029.668141] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8029.676241] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8029.702141] Lustre: Skipped 1 previous similar message [ 8029.708120] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8029.713931] Lustre: Skipped 2 previous similar messages [ 8035.416588] Lustre: server umount lustre-MDT0000 complete [ 8036.832407] LustreError: 229039:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8036.845170] LustreError: 229039:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8039.700786] LustreError: 229022:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786728498 with bad export cookie 5028852174911422787 [ 8039.720274] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8039.732751] LustreError: 229022:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8040.252245] Lustre: server umount lustre-MDT0001 complete [ 8051.999982] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 8054.500517] Key type lgssc unregistered [ 8054.853762] LNet: 234275:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8054.867359] LNetError: 234275:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8054.897655] LNet: Removed LNI 192.168.202.134@tcp [ 8056.066393] Key type .llcrypt unregistered [ 8056.067647] Key type ._llcrypt unregistered [ 8094.234372] Key type ._llcrypt registered [ 8094.237517] Key type .llcrypt registered [ 8094.389360] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 8095.442593] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8095.469577] alg: No test for adler32 (adler32-zlib) [ 8096.689606] Lustre: Lustre: Build Version: 2.17.57_1_g689e325 [ 8096.979356] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 8098.671256] Key type lgssc registered [ 8099.883448] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8108.971655] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8109.000094] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8110.555877] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8114.731886] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 8120.813026] LustreError: 236005:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8120.849864] LustreError: 236005:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 8125.943469] LustreError: 236006:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8126.143156] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8126.815514] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8133.065802] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 8137.963257] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 8149.102305] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8149.761194] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8155.970637] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 8157.315761] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 8166.178742] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8166.405924] Lustre: lustre-OST0001: new disk, initializing [ 8166.418747] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8166.425794] Lustre: 238325:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8166.587239] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8173.078120] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 8173.759841] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8173.768610] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8173.906552] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8184.506800] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8192.592380] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 8208.874106] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8208.885285] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8208.905848] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8209.889816] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8209.891951] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8209.927030] Lustre: Skipped 2 previous similar messages [ 8209.930576] Lustre: Skipped 2 previous similar messages [ 8211.407770] Lustre: server umount lustre-MDT0000 complete [ 8215.009410] LustreError: 236734:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8215.052671] LustreError: 236734:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 8215.566721] LustreError: 239092:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786728674 with bad export cookie 770980311814776403 [ 8215.568327] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8215.585106] LustreError: 239092:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8216.031536] Lustre: server umount lustre-MDT0001 complete [ 8231.103958] Lustre: server umount lustre-OST0000 complete [ 8246.779300] Lustre: server umount lustre-OST0001 complete [ 8279.652693] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 8290.633739] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8291.094833] LustreError: 241444:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8291.206614] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8296.415759] LustreError: 241445:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8296.477626] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 8306.659778] LustreError: 241445:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8306.671241] LustreError: 241445:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 8307.198941] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8307.552660] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8314.549137] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 8318.150317] Lustre: 242587:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8327.496137] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8327.801907] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8332.937644] LustreError: 242940:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8336.062066] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49760) [ 8336.243977] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 8346.785989] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8355.845166] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 8364.942774] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8368.658360] Lustre: 244458:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8382.945996] Lustre: mylustre-MDT0000-lwp-OST0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8382.948348] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8382.963507] Lustre: Skipped 3 previous similar messages [ 8382.972593] Lustre: Skipped 3 previous similar messages [ 8388.067584] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8388.079749] Lustre: Skipped 4 previous similar messages [ 8389.299584] Lustre: server umount mylustre-MDT0000 complete [ 8393.193517] LustreError: 241440:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8393.231784] LustreError: 241440:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 8 previous similar messages [ 8394.210990] LustreError: 241421:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786728853 with bad export cookie 770980311814779203 [ 8394.218459] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8394.228285] LustreError: 241421:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8394.775479] Lustre: server umount mylustre-MDT0001 complete [ 8408.448253] Lustre: server umount mylustre-OST0000 complete [ 8424.201910] Lustre: server umount mylustre-OST0001 complete [ 8449.348824] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 8460.060142] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8460.492777] LustreError: 246770:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8460.574225] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8460.582576] Lustre: Skipped 1 previous similar message [ 8464.788626] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 8474.876527] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8479.934729] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 8483.477974] Lustre: 247912:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8491.301414] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8499.054589] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 8505.218466] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49792) [ 8508.414582] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8514.042124] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 8514.499605] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 8522.277809] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8525.691662] Lustre: 249781:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8534.506617] Lustre: tfs-MDT0000-lwp-OST0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8534.515877] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 8534.520659] Lustre: Skipped 2 previous similar messages [ 8534.539485] Lustre: Skipped 3 previous similar messages [ 8535.807414] Lustre: server umount tfs-MDT0000 complete [ 8539.698400] LustreError: 246745:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786728998 with bad export cookie 770980311814780876 [ 8539.714394] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8539.723904] LustreError: 246745:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8540.327925] Lustre: server umount tfs-MDT0001 complete [ 8555.090145] Lustre: server umount tfs-OST0000 complete [ 8569.426688] Lustre: server umount tfs-OST0001 complete [ 8591.305782] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 8602.511087] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8602.869805] LustreError: 252031:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8602.887812] LustreError: 252031:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 9 previous similar messages [ 8602.951180] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8602.956420] Lustre: Skipped 3 previous similar messages [ 8608.119646] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 8619.425194] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8625.178987] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 8628.593453] Lustre: 253172:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8635.136428] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8641.837569] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 8647.786722] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49824) [ 8650.215676] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8655.896230] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 8655.962146] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 8663.438821] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8667.543784] Lustre: 255044:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8674.792529] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 13:38:51 (1786729131) [ 8691.692756] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8691.699243] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8691.703794] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8691.741843] Lustre: Skipped 4 previous similar messages [ 8696.618798] Lustre: server umount lustre-MDT0000 complete [ 8700.005886] LustreError: 252008:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786729158 with bad export cookie 770980311814782542 [ 8700.017742] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8700.022230] LustreError: 252008:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8701.922593] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8701.953837] Lustre: Skipped 1 previous similar message [ 8706.543498] Lustre: server umount lustre-MDT0001 complete [ 8716.898794] Lustre: server umount lustre-OST0000 complete [ 8727.065424] Lustre: server umount lustre-OST0001 complete [ 8734.554384] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 8746.720950] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 8785.374850] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 8785.374850] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8785.374850] [ 8785.408866] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8785.669894] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8785.724573] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8785.826363] Lustre: lustre-MDT0000: new disk, initializing [ 8785.900243] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8785.907870] Lustre: Skipped 3 previous similar messages [ 8785.922629] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8790.782241] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 8802.219079] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 8802.219079] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8802.219079] [ 8802.257609] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8802.462501] Lustre: 258227:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8802.603913] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8802.606735] Lustre: Skipped 1 previous similar message [ 8802.772280] Lustre: lustre-MDT0001: new disk, initializing [ 8802.973533] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8802.983504] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8807.218904] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 8819.358701] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8819.604624] Lustre: lustre-OST0000: new disk, initializing [ 8819.607774] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8819.615731] Lustre: 259813:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8820.888450] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8820.903601] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8821.146801] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8826.565888] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 8836.766491] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8837.047232] Lustre: lustre-OST0001: new disk, initializing [ 8837.054096] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8837.066065] Lustre: 260682:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8838.211540] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8838.226471] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8838.365936] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8843.392836] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 8854.600553] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 13:41:51 (1786729311) [ 8867.690357] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 13:42:04 (1786729324) [ 8882.657633] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8882.667571] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8882.672014] LustreError: Skipped 1 previous similar message [ 8882.675526] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8882.690720] Lustre: Skipped 1 previous similar message [ 8882.718287] Lustre: Skipped 8 previous similar messages [ 8888.405856] Lustre: server umount lustre-OST0000 complete [ 8889.831407] LustreError: 260744:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8889.872865] LustreError: 260744:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 15 previous similar messages [ 8894.954373] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8898.866193] Lustre: server umount lustre-MDT0000 complete [ 8903.088482] LustreError: 258220:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786729361 with bad export cookie 770980311814784096 [ 8903.094828] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8903.105296] LustreError: 258220:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8910.168908] Lustre: server umount lustre-MDT0001 complete [ 8924.035440] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 9000.832466] Lustre: server umount lustre-OST0001 complete [ 9006.956932] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 9024.118685] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 9064.459271] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9064.720511] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9064.751494] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9064.844697] Lustre: lustre-MDT0000: new disk, initializing [ 9064.934845] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9064.943713] Lustre: Skipped 3 previous similar messages [ 9064.960401] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9070.846714] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 9082.236652] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9082.361116] Lustre: 265479:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9082.405259] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9082.408976] Lustre: Skipped 1 previous similar message [ 9082.464362] Lustre: lustre-MDT0001: new disk, initializing [ 9082.593178] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9082.601638] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9087.567650] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 9097.452624] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9103.933918] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9111.538223] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9111.818867] Lustre: lustre-OST0000: new disk, initializing [ 9111.825078] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9111.839561] Lustre: 267449:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9113.621949] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9113.638112] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9113.790458] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9118.082390] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 9127.705217] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9134.564098] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9134.569450] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9134.574014] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9134.586393] Lustre: Skipped 5 previous similar messages [ 9134.595986] Lustre: Skipped 8 previous similar messages [ 9139.412623] Lustre: server umount lustre-OST0000 complete [ 9144.288813] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9144.295331] LustreError: Skipped 1 previous similar message [ 9157.089131] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9157.288937] Lustre: server umount lustre-MDT0000 complete [ 9161.997348] LustreError: 265472:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786729620 with bad export cookie 770980311814785566 [ 9162.035721] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9162.628933] Lustre: server umount lustre-MDT0001 complete [ 9176.007853] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 9179.052713] Key type lgssc unregistered [ 9179.342816] LNet: 269217:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9179.347679] LNetError: 269217:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9179.357995] LNet: Removed LNI 192.168.202.134@tcp [ 9180.403337] Key type .llcrypt unregistered [ 9180.406390] Key type ._llcrypt unregistered [ 9212.212054] Key type ._llcrypt registered [ 9212.213861] Key type .llcrypt registered [ 9212.402390] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 9213.707973] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9213.851787] alg: No test for adler32 (adler32-zlib) [ 9215.178678] Lustre: Lustre: Build Version: 2.17.57_1_g689e325 [ 9215.614797] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 9217.408615] Key type lgssc registered [ 9219.098076] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9228.091461] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9228.129775] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9229.739750] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9234.480887] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 9240.035474] LustreError: 270680:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9240.057536] LustreError: 270680:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 9242.715546] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9243.232592] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9247.616162] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 9255.015605] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9260.778235] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9267.349679] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9267.641932] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9274.307370] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 9274.820907] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 9281.887821] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9283.134940] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 9285.091877] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9285.098261] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9285.104106] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9285.110967] Lustre: Skipped 1 previous similar message [ 9290.209883] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9290.224991] Lustre: Skipped 2 previous similar messages [ 9291.143286] Lustre: server umount lustre-OST0000 complete [ 9295.337275] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9295.342589] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9295.357101] Lustre: Skipped 1 previous similar message [ 9300.449305] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9300.458301] Lustre: Skipped 2 previous similar messages [ 9300.966916] Lustre: server umount lustre-MDT0000 complete [ 9305.066434] LustreError: 270660:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786729763 with bad export cookie 12574527900419494757 [ 9305.071076] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9305.078216] LustreError: 270660:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9305.393260] Lustre: server umount lustre-MDT0001 complete [ 9316.901396] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 9322.968333] Key type lgssc unregistered [ 9323.239871] LNet: 274267:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9323.266400] LNetError: 274267:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9323.285422] LNet: Removed LNI 192.168.202.134@tcp [ 9324.108835] Key type .llcrypt unregistered [ 9324.118131] Key type ._llcrypt unregistered [ 9347.034553] Key type ._llcrypt registered [ 9347.036368] Key type .llcrypt registered [ 9347.157199] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 9348.212760] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9348.267573] alg: No test for adler32 (adler32-zlib) [ 9349.604374] Lustre: Lustre: Build Version: 2.17.57_1_g689e325 [ 9349.797207] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 9351.440224] Key type lgssc registered [ 9352.917697] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9361.639041] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9361.658403] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9377.312568] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 9382.431759] LustreError: 275463:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.134@tcp: failed processing log, type 4: rc = -110 [ 9409.120164] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9416.869654] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 9426.831644] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9429.784610] Lustre: server umount lustre-OST0000 complete [ 9450.981243] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 9454.280665] Key type lgssc unregistered [ 9454.639678] LNet: 277100:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9454.654694] LNetError: 277100:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9454.683151] LNet: Removed LNI 192.168.202.134@tcp [ 9455.605290] Key type .llcrypt unregistered [ 9455.610848] Key type ._llcrypt unregistered [ 9498.839730] Key type ._llcrypt registered [ 9498.841931] Key type .llcrypt registered [ 9498.936654] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 9499.968748] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9500.021835] alg: No test for adler32 (adler32-zlib) [ 9501.669117] Lustre: Lustre: Build Version: 2.17.57_1_g689e325 [ 9502.322370] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 9504.223388] Key type lgssc registered [ 9507.142099] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9527.703038] loop0: detected capacity change from 0 to 204800000 [ 9528.026356] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9528.051359] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9528.561103] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9530.020984] loop0: detected capacity change from 0 to 204800000 [ 9530.187042] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9530.195149] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9530.852743] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9532.435475] loop0: detected capacity change from 0 to 204800000 [ 9532.724541] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9532.729052] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9532.949716] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9534.323608] loop0: detected capacity change from 0 to 204800000 [ 9534.672342] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9534.678920] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9534.748940] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9535.457898] loop0: detected capacity change from 0 to 204800000 [ 9535.498934] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9540.898644] loop0: detected capacity change from 0 to 204800000 [ 9540.956051] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9544.658345] loop0: detected capacity change from 0 to 204800000 [ 9544.713630] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9549.348907] loop0: detected capacity change from 0 to 204800000 [ 9549.397815] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9554.732090] loop0: detected capacity change from 0 to 204800000 [ 9554.863299] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9554.921392] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9554.987328] Lustre: 280513:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9554.994373] Lustre: 280513:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9555.002580] Lustre: 280513:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 9555.010310] Lustre: 280513:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9555.018947] Lustre: 280513:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 9555.026968] Lustre: 280513:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9555.035897] Lustre: 280513:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 9555.044839] Lustre: 280513:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 9555.053248] Lustre: 280513:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 9555.061637] Lustre: 280513:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 9555.075623] Lustre: 280513:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 9555.088515] Lustre: 280513:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 9555.103481] Lustre: 280513:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 9555.118984] Lustre: 280513:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 9561.613784] Lustre: server umount MGS complete [ 9562.465788] loop0: detected capacity change from 0 to 204800000 [ 9562.576523] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9563.130196] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9563.202368] Lustre: lustre-MDT0000: Aborting client recovery [ 9564.170304] loop1: detected capacity change from 0 to 204800000 [ 9564.275710] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9564.371798] Lustre: 280947:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9564.389381] Lustre: 280947:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9564.412463] Lustre: 280947:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 9564.425744] Lustre: 280947:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9564.447865] Lustre: 280947:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9564.459318] Lustre: 280947:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 9564.471990] Lustre: 280947:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 9564.499471] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9564.654667] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9564.707188] Lustre: lustre-MDT0001: Aborting client recovery [ 9565.555125] loop2: detected capacity change from 0 to 204800000 [ 9565.682316] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9565.796812] Lustre: 281026:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9565.810793] Lustre: 281026:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9565.823746] Lustre: 281026:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9565.846957] Lustre: 281026:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9565.864627] Lustre: 281026:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9565.872806] Lustre: 281026:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9565.879280] Lustre: 281026:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9566.044049] Lustre: Found index 0 for lustre-OST0000, updating log [ 9566.074317] Lustre: 281045:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9566.125274] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9566.142651] Lustre: lustre-OST0000: Aborting recovery [ 9567.516337] loop3: detected capacity change from 0 to 204800000 [ 9567.718882] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9567.860785] Lustre: 281110:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9567.871465] Lustre: 281110:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9567.882357] Lustre: 281110:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9567.896802] Lustre: 281110:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9567.907739] Lustre: 281110:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9567.927510] Lustre: 281110:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9567.937237] Lustre: 281110:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9567.974173] Lustre: Found index 1 for lustre-OST0001, updating log [ 9568.020204] Lustre: 281117:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9568.090873] Lustre: lustre-OST0001: Aborting recovery [ 9572.202087] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9572.208781] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9572.233049] LustreError: 280891:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 9, retries 0, failed: rc = -108 [ 9572.281798] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 9572.288420] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 9574.275122] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 9574.291286] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 9576.508666] LustreError: 280858:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 3560us: rc = -2 [ 9577.074158] LustreError: 280859:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1169us: rc = -2 [ 9577.090720] LustreError: 280859:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 21 previous similar messages [ 9590.239906] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9590.250704] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9590.288491] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9594.864434] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9594.882261] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9594.891500] Lustre: Skipped 2 previous similar messages [ 9594.906297] Lustre: Skipped 3 previous similar messages [ 9596.354117] Lustre: server umount lustre-MDT0000 complete [ 9597.549723] LustreError: 281116:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786730056 with bad export cookie 2332609081701873160 [ 9597.567392] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9597.580779] LustreError: 281116:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9599.970349] LustreError: 280857:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9599.972855] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9599.973326] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9600.007611] LustreError: 280857:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 9600.035081] Lustre: Skipped 1 previous similar message [ 9602.976491] LustreError: 281125:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9602.977295] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9603.010716] Lustre: Skipped 2 previous similar messages [ 9604.576653] Lustre: server umount lustre-MDT0001 complete [ 9611.945106] Lustre: server umount lustre-OST0000 complete [ 9619.600519] Lustre: server umount lustre-OST0001 complete [ 9630.502845] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 13:54:46 (1786730086) [ 9672.868484] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 9683.751859] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 9724.668788] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9724.857104] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9724.868399] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9724.904637] Lustre: lustre-MDT0000: new disk, initializing [ 9724.958439] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9724.963117] Lustre: Skipped 1 previous similar message [ 9724.976254] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9730.422370] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 9740.621869] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9740.763977] Lustre: 285192:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9740.813704] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9740.817587] Lustre: Skipped 1 previous similar message [ 9740.894566] Lustre: lustre-MDT0001: new disk, initializing [ 9741.001545] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9741.046629] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9741.054991] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9745.794077] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 9755.877850] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9762.763866] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9770.658827] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9770.892733] Lustre: lustre-OST0000: new disk, initializing [ 9770.896139] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9770.904987] Lustre: 287162:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9771.008479] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9776.966925] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 9780.305855] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9780.327405] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9780.420559] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9787.028960] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9793.381773] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9805.619986] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9814.690425] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9857.505843] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9857.508252] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9857.509697] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9857.516623] LustreError: Skipped 1 previous similar message [ 9857.545763] Lustre: Skipped 1 previous similar message [ 9859.390896] Lustre: server umount lustre-OST0000 complete [ 9864.160385] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9864.169088] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9867.744586] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9867.760284] Lustre: Skipped 3 previous similar messages [ 9869.542108] Lustre: server umount lustre-MDT0000 complete [ 9872.885851] LustreError: 285199:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9872.931476] LustreError: 285199:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 9874.673858] LustreError: 285183:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786730333 with bad export cookie 2332609081701875869 [ 9874.687558] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9874.696968] LustreError: 285183:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9875.234185] Lustre: server umount lustre-MDT0001 complete [ 9886.849728] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9893.206796] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 9899.486571] Lustre: server umount MGS complete [ 9912.871835] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9913.544414] LustreError: 291074:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9913.586974] LustreError: 291087:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9913.656095] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9917.828370] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 9918.944556] LustreError: 291075:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9926.684348] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9932.055318] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 9940.255243] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9947.907166] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9956.567157] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9957.280976] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9957.303272] Lustre: Skipped 1 previous similar message [ 9967.479496] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 9975.741425] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10004.455281] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10004.471513] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10004.499442] Lustre: Skipped 1 previous similar message [10004.502389] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10017.247405] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10017.593536] Lustre: server umount lustre-OST0000 complete [10022.893040] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10022.905593] Lustre: Skipped 1 previous similar message [10036.191787] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10036.374653] Lustre: server umount lustre-MDT0000 complete [10037.218034] LustreError: 291075:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10037.230305] LustreError: 291075:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10040.099489] LustreError: 293910:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786730498 with bad export cookie 2332609081701877675 [10040.114666] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10040.122562] LustreError: 293910:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10040.495699] Lustre: server umount lustre-MDT0001 complete [10053.316731] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [10055.990509] Key type lgssc unregistered [10056.308980] LNet: 295036:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10056.317993] LNetError: 295036:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10056.336814] LNet: Removed LNI 192.168.202.134@tcp [10057.752742] Key type .llcrypt unregistered [10057.755436] Key type ._llcrypt unregistered [10095.439142] Key type ._llcrypt registered [10095.440846] Key type .llcrypt registered [10095.652350] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [10118.239462] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [10119.491140] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10119.585199] alg: No test for adler32 (adler32-zlib) [10120.955219] Lustre: Lustre: Build Version: 2.17.57_1_g689e325 [10121.350717] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [10123.247258] Key type lgssc registered [10124.240644] Lustre: Echo OBD driver; http://www.lustre.org/ [10168.798332] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10168.829665] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10170.339256] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10170.402666] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10170.692058] Lustre: lustre-MDT0000: new disk, initializing [10170.850995] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10170.887215] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10177.621022] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [10190.663579] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10190.846290] Lustre: 298997:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10190.967817] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10190.974912] Lustre: Skipped 1 previous similar message [10191.068956] Lustre: lustre-MDT0001: new disk, initializing [10191.205431] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10191.275412] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10191.305515] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10197.105302] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [10208.513655] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10215.181623] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10222.010199] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10222.381814] Lustre: lustre-OST0000: new disk, initializing [10222.385362] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10222.394054] Lustre: 300966:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10222.567497] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10230.585227] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [10231.508072] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10231.537164] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10231.791916] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10245.700603] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10257.251831] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10269.442916] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10278.137883] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10282.995829] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [10329.575519] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10329.582826] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10329.602351] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10334.187038] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10334.200797] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10334.218478] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10335.593903] Lustre: server umount lustre-OST0000 complete [10344.416537] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10344.426723] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10344.429772] Lustre: Skipped 2 previous similar messages [10349.564733] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10349.572812] Lustre: Skipped 1 previous similar message [10354.664689] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10354.674609] Lustre: Skipped 1 previous similar message [10356.192272] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10356.700715] Lustre: server umount lustre-MDT0000 complete [10363.072294] LustreError: 300965:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786730821 with bad export cookie 17284875943420260723 [10363.074751] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10363.092893] LustreError: 300965:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10364.024958] Lustre: server umount lustre-MDT0001 complete [10378.272290] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10384.707984] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [10390.591768] Lustre: server umount MGS complete [10403.036913] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10403.427810] LustreError: 304892:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10403.452555] LustreError: 304892:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [10403.460171] LustreError: 304905:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10403.550584] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10407.794555] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [10408.929152] LustreError: 304893:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10414.049320] LustreError: 304892:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10417.211388] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10418.094542] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10424.055348] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [10435.140435] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10442.760383] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10451.306297] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10451.516305] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10458.621119] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [10467.092655] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10513.889687] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10513.907490] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10513.928024] Lustre: Skipped 1 previous similar message [10513.941731] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10513.951086] Lustre: Skipped 2 previous similar messages [10517.622824] Lustre: server umount lustre-OST0000 complete [10525.162128] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10525.181395] Lustre: Skipped 2 previous similar messages [10528.446309] Lustre: server umount lustre-MDT0000 complete [10530.285983] LustreError: 304892:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10530.322090] LustreError: 304892:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10532.250188] LustreError: 307643:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786730991 with bad export cookie 17284875943420262676 [10532.258940] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10532.262326] LustreError: 307643:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [10532.815849] Lustre: server umount lustre-MDT0001 complete [10548.838195] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [10552.368323] Key type lgssc unregistered [10552.816639] LNet: 308863:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10552.829298] LNetError: 308863:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10552.863536] LNet: Removed LNI 192.168.202.134@tcp [10554.509093] Key type .llcrypt unregistered [10554.516255] Key type ._llcrypt unregistered [10579.604815] Key type ._llcrypt registered [10579.610156] Key type .llcrypt registered [10579.790437] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [10599.258772] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [10601.195904] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10601.365782] alg: No test for adler32 (adler32-zlib) [10602.515629] Lustre: Lustre: Build Version: 2.17.57_1_g689e325 [10602.751915] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [10604.399236] Key type lgssc registered [10606.248599] Lustre: Echo OBD driver; http://www.lustre.org/ [10637.776555] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [10653.005379] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10653.078499] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10654.430350] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10654.483981] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10654.659420] Lustre: lustre-MDT0000: new disk, initializing [10654.882028] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10654.936913] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10659.658181] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [10673.231970] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10673.533877] Lustre: lustre-OST0000: new disk, initializing [10673.537476] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10673.542610] Lustre: Skipped 1 previous similar message [10673.546463] Lustre: 312916:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10673.614858] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10674.044145] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [10674.083346] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [10674.230961] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [10682.608950] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [11636.466297] Lustre: server umount lustre-MDT0000 complete [11640.383570] LustreError: 312104:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786732099 with bad export cookie 8494629926421278561 [11640.390517] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11650.776452] Lustre: server umount lustre-OST0000 complete [11680.366103] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [11692.276944] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [11731.788577] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11732.032039] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11732.056759] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11732.116405] Lustre: lustre-MDT0000: new disk, initializing [11732.168498] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11732.182426] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11737.165358] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [11748.258529] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11748.397766] Lustre: 317271:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11748.508087] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11748.514402] Lustre: Skipped 1 previous similar message [11748.617826] Lustre: lustre-MDT0001: new disk, initializing [11748.737726] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11748.795342] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11748.812044] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11753.790819] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [11763.970654] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11771.003819] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11777.314464] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11777.556146] Lustre: lustre-OST0000: new disk, initializing [11777.561887] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11777.566909] Lustre: 319241:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11777.649101] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11782.695733] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11782.707585] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11782.770235] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11783.778924] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [11794.214034] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11798.759602] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11798.920725] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11802.893777] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11803.140335] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11808.236053] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11808.241542] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11808.247845] Lustre: Skipped 1 previous similar message [11808.268177] Lustre: Skipped 1 previous similar message [11811.721527] Lustre: server umount lustre-OST0000 complete [11818.465799] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11818.468571] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11818.488756] Lustre: Skipped 1 previous similar message [11818.499349] Lustre: Skipped 1 previous similar message [11822.174226] Lustre: server umount lustre-MDT0000 complete [11826.484632] LustreError: 317264:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786732285 with bad export cookie 8494629926424219380 [11826.508735] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11827.014464] Lustre: server umount lustre-MDT0001 complete [11831.143578] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [11833.175316] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 14:31:29 (1786732289) [11841.985545] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11842.523085] LustreError: 321555:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11842.546591] LustreError: 321555:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [11842.610778] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11847.662600] LustreError: 321556:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11848.361739] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [11852.776598] LustreError: 321555:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11858.916678] LustreError: 321555:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11858.958028] LustreError: 321555:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11859.291731] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11859.875764] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11866.208029] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [11875.385458] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11881.717428] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11889.542743] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11889.921357] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11897.896893] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [11906.968760] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11913.576683] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11913.695625] Lustre: lustre-OST0001: new disk, initializing [11913.699082] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [11913.704582] Lustre: 324392:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11913.788438] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11919.503829] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [11919.516932] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [11919.558861] LustreError: 324723:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11919.674386] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [11920.615147] LustreError: 324736:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11920.631811] LustreError: 324736:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [11921.662371] LustreError: 324723:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11923.757938] LustreError: 324736:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11923.773131] LustreError: 324736:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 4 previous similar messages [11927.867200] LustreError: 324723:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11927.878735] LustreError: 324723:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 6 previous similar messages [11931.469465] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11931.683628] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11936.167089] LustreError: 324723:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11936.176772] LustreError: 324723:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [11937.795857] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9303d0381800.ost_server_uuid 50 [11939.678189] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9303d0381800.ost_server_uuid in FULL state after 0 sec [11945.511749] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9303d0381800.ost_server_uuid 50 [11947.516884] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9303d0381800.ost_server_uuid in IDLE state after 0 sec [11949.628921] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [11964.896623] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11964.900797] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11964.907996] LustreError: Skipped 1 previous similar message [11964.908707] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11964.921872] Lustre: Skipped 1 previous similar message [11969.408387] Lustre: server umount lustre-OST0001 complete [11970.929527] LustreError: 324363:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.202.34@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11976.163032] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11976.199694] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11976.202630] Lustre: Skipped 1 previous similar message [11976.220633] Lustre: Skipped 5 previous similar messages [11981.286435] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11981.312745] LustreError: 323378:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11981.329786] LustreError: 323378:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [11981.851321] Lustre: server umount lustre-OST0000 complete [11986.406731] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11991.524805] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11991.532374] Lustre: Skipped 4 previous similar messages [11992.028233] Lustre: server umount lustre-MDT0000 complete [11996.159069] LustreError: 321537:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786732455 with bad export cookie 8494629926424220262 [11996.169113] LustreError: 321537:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11996.175114] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11996.562026] Lustre: server umount lustre-MDT0001 complete [12011.065919] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [12014.385997] Key type lgssc unregistered [12014.883413] LNet: 327083:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12014.907655] LNetError: 327083:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12014.953543] LNet: Removed LNI 192.168.202.134@tcp [12015.988116] Key type .llcrypt unregistered [12015.989794] Key type ._llcrypt unregistered [12047.156418] Key type ._llcrypt registered [12047.162992] Key type .llcrypt registered [12047.287612] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [12048.899148] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12048.943944] alg: No test for adler32 (adler32-zlib) [12050.015498] Lustre: Lustre: Build Version: 2.17.57_1_g689e325 [12050.383159] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [12052.136540] Key type lgssc registered [12053.402632] Lustre: Echo OBD driver; http://www.lustre.org/ [12061.338491] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12061.371945] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12062.683162] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12065.638172] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [12072.341753] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12072.937772] LustreError: 328505:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12072.968256] LustreError: 328505:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12078.054177] LustreError: 328506:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12079.523918] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12079.880729] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12083.932796] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [12092.677200] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12093.022783] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12097.890516] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [12103.916958] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12104.298051] LustreError: 330186:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12107.376412] LustreError: 330185:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12107.409560] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [12107.411722] LustreError: 330185:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12109.750501] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12109.927172] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12114.954123] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [12115.631922] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [12122.807465] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12130.296171] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [12130.510481] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12232.687174] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12232.690476] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12232.699632] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12232.707559] Lustre: Skipped 1 previous similar message [12236.624590] Lustre: server umount lustre-OST0001 complete [12237.801704] LustreError: 330186:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12237.827557] LustreError: 330186:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12242.916689] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12242.917189] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12242.923776] LustreError: Skipped 1 previous similar message [12242.924473] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12242.924478] Lustre: Skipped 1 previous similar message [12242.943769] Lustre: Skipped 1 previous similar message [12248.040575] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12248.043531] LustreError: 330853:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12248.052823] Lustre: Skipped 2 previous similar messages [12248.086573] LustreError: 330853:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [12248.765109] Lustre: server umount lustre-OST0000 complete [12253.155641] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12253.174850] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12253.182868] Lustre: Skipped 1 previous similar message [12253.190975] Lustre: Skipped 1 previous similar message [12258.274199] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12258.281475] Lustre: Skipped 1 previous similar message [12258.783729] Lustre: server umount lustre-MDT0000 complete [12262.380838] LustreError: 332191:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786732721 with bad export cookie 3231759369167049730 [12262.384254] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12262.389112] LustreError: 332191:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12262.644743] Lustre: server umount lustre-MDT0001 complete [12275.191412] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [12278.490636] Key type lgssc unregistered [12278.779789] LNet: 333450:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12278.786261] LNetError: 333450:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12278.808274] LNet: Removed LNI 192.168.202.134@tcp [12279.932346] Key type .llcrypt unregistered [12279.934400] Key type ._llcrypt unregistered [12324.569638] Key type ._llcrypt registered [12324.571829] Key type .llcrypt registered [12324.664859] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [12325.461723] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12325.480750] alg: No test for adler32 (adler32-zlib) [12326.577404] Lustre: Lustre: Build Version: 2.17.57_1_g689e325 [12326.860583] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [12328.503180] Key type lgssc registered [12329.328444] Lustre: Echo OBD driver; http://www.lustre.org/ [12338.479306] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12338.495975] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12339.899123] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12344.609510] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [12349.410072] LustreError: 335740:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12349.428157] LustreError: 335740:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12353.687679] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12354.003117] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12358.184480] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [12360.782784] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [12367.220510] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12367.764858] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12373.672678] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [12379.049327] LustreError: 337229:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12383.518059] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12383.888282] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12384.960637] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [12384.966790] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [12390.902384] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [12400.244840] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12407.572826] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [12420.576881] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12420.590994] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12420.611766] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12421.095098] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12421.114385] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12421.122730] Lustre: Skipped 2 previous similar messages [12421.146443] Lustre: Skipped 3 previous similar messages [12426.224162] LustreError: 335740:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12426.248980] LustreError: 335740:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [12426.528593] Lustre: server umount lustre-MDT0000 complete [12430.493677] LustreError: 335721:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786732889 with bad export cookie 239233892724273728 [12430.507187] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12430.508430] LustreError: 335721:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12430.845654] Lustre: server umount lustre-MDT0001 complete [12444.891342] Lustre: server umount lustre-OST0000 complete [12447.716778] Lustre: 335001:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786732890/real 1786732890] req@ffff9b2082a8df80 x1873525116826624/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786732906 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [12447.770926] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12448.572336] Lustre: server umount lustre-OST0001 complete [12463.367171] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [12472.389292] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12472.724031] LustreError: 341004:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12472.848318] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12477.302904] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [12477.926872] LustreError: 341005:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12485.071697] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12485.406351] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12489.356193] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [12492.377047] Lustre: 342147:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [12498.833168] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12504.921272] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [12510.500197] LustreError: 342499:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12510.514214] LustreError: 342499:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12513.264688] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12513.575852] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12513.579981] Lustre: Skipped 1 previous similar message [12517.753313] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [12517.759193] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [12520.490961] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [12527.807715] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12531.336470] Lustre: 344015:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [12536.800892] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12536.806044] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12536.817126] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12542.058096] Lustre: server umount lustre-MDT0000 complete [12543.459324] LustreError: 341004:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12543.491200] LustreError: 341004:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [12545.776377] LustreError: 340985:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786733004 with bad export cookie 239233892724275429 [12545.782936] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12545.789961] LustreError: 340985:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12546.116990] Lustre: server umount lustre-MDT0001 complete [12559.361208] Lustre: server umount lustre-OST0000 complete [12573.094970] Lustre: server umount lustre-OST0001 complete [12576.302582] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [12577.756484] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [12579.312471] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 14:43:56 (1786733036) [12600.877981] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [12609.295785] loop0: detected capacity change from 0 to 1099511627776 [12609.843487] SGI XFS with ACLs, security attributes, quota, no debug enabled [12609.856591] *** VALIDATE xfs *** [12609.877322] XFS (loop0): Mounting V5 Filesystem [12609.902155] XFS (loop0): Ending clean mount [12611.300559] loop1: detected capacity change from 0 to 18691697672192 [12725.170885] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [12737.185700] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [12774.299364] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12774.572771] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12774.617692] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12774.686340] Lustre: lustre-MDT0000: new disk, initializing [12774.795124] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12774.809476] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12778.980481] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [12788.791497] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12788.952844] Lustre: 349157:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12789.009053] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12789.017658] Lustre: Skipped 1 previous similar message [12789.087182] Lustre: lustre-MDT0001: new disk, initializing [12789.239462] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12789.254793] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12792.975931] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [12801.699546] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12806.895236] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12813.058503] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12813.336581] Lustre: lustre-OST0000: new disk, initializing [12813.339972] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12813.346310] Lustre: 351124:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12814.750751] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12814.764508] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12814.873449] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12819.236626] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [12831.596844] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12836.465966] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12836.636678] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12841.165155] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [12841.447475] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12845.540309] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12845.550025] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12845.571747] Lustre: Skipped 4 previous similar messages [12845.583419] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12845.602898] Lustre: Skipped 5 previous similar messages [12849.721270] Lustre: server umount lustre-OST0000 complete [12855.776157] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12855.780546] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12855.801913] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12855.812221] Lustre: Skipped 1 previous similar message [12860.307313] Lustre: server umount lustre-MDT0000 complete [12860.897451] LustreError: 349967:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12860.917171] LustreError: 349967:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12864.437297] LustreError: 349962:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786733323 with bad export cookie 239233892724276983 [12864.441250] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12864.449705] LustreError: 349962:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12864.776979] Lustre: server umount lustre-MDT0001 complete [12872.921463] XFS (loop0): Unmounting Filesystem [12878.609615] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 14:48:55 (1786733335) [12885.388234] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12885.843123] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12885.848700] Lustre: Skipped 2 previous similar messages [12889.406780] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [12896.827384] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12901.173607] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [12908.629419] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12914.912050] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12922.064311] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12928.616429] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [12935.711227] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12943.844190] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12943.853457] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12943.869193] Lustre: Skipped 1 previous similar message [12943.880367] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12949.698713] Lustre: server umount lustre-OST0000 complete [12953.569592] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12953.583491] LustreError: Skipped 1 previous similar message [12961.249554] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12961.252449] Lustre: Skipped 6 previous similar messages [12967.391573] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12967.725120] Lustre: server umount lustre-MDT0000 complete [12970.472214] LustreError: 353678:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12970.494062] LustreError: 353678:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [12971.123678] LustreError: 354420:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786733430 with bad export cookie 239233892724277865 [12971.131065] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12971.134820] LustreError: 354420:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12971.433858] Lustre: server umount lustre-MDT0001 complete [12981.738435] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [12984.153868] Key type lgssc unregistered [12984.390721] LNet: 357316:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12984.402468] LNetError: 357316:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12985.447932] LNet: Removed LNI 192.168.202.134@tcp [12986.395257] Key type .llcrypt unregistered [12986.397109] Key type ._llcrypt unregistered [13013.020858] Key type ._llcrypt registered [13013.024837] Key type .llcrypt registered [13013.165883] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [13015.072578] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [13015.111974] alg: No test for adler32 (adler32-zlib) [13016.267366] Lustre: Lustre: Build Version: 2.17.57_1_g689e325 [13016.617902] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [13018.295268] Key type lgssc registered [13019.204545] Lustre: Echo OBD driver; http://www.lustre.org/ [13026.107524] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [13026.124536] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13027.751833] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13032.996399] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [13038.054097] LustreError: 358884:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13038.082220] LustreError: 358884:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [13040.671088] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13040.919566] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13045.090675] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [13053.932830] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13060.103145] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13069.241260] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13069.751631] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13078.824764] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [13087.252227] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13094.289845] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.34@tcp (stopping) [13096.421332] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13096.461745] Lustre: Skipped 1 previous similar message [13096.463035] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13096.491945] Lustre: Skipped 1 previous similar message [13099.380803] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.34@tcp (stopping) [13100.513995] Lustre: server umount lustre-MDT0001 complete [13101.540675] LustreError: 358879:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13101.561962] LustreError: 358879:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [13104.488727] LustreError: 360425:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.34@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13106.658271] LustreError: 358883:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13106.675978] LustreError: 358883:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [13111.799177] LustreError: 358884:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13111.812239] LustreError: 358884:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [13112.001251] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13112.082341] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13112.088030] Lustre: Found index 1 for lustre-MDT0001, updating log [13112.117329] Lustre: 358874:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13112.357306] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13116.968739] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [13117.422483] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13120.019981] LustreError: 362484:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13120.036681] LustreError: 362484:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13120.042951] LustreError: 358876:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13120.087555] LustreError: 362487:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.134@tcp: cfg command failed: rc = -17 [13120.094197] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [13120.094197] [13424.613598] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13424.622527] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13424.625672] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13429.730755] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13429.749146] Lustre: Skipped 3 previous similar messages [13430.549581] Lustre: server umount lustre-MDT0001 complete [13432.174371] LustreError: 363971:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.34@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13440.640801] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13440.765731] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13440.771875] Lustre: Found index 1 for lustre-MDT0001, updating log [13440.876390] Lustre: 358873:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13441.299841] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13445.855812] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [13446.658062] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13447.676853] LustreError: 370737:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13447.702233] LustreError: 370737:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13447.727330] LustreError: 358876:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13447.745399] LustreError: 358876:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13447.809168] LustreError: 370742:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [13447.815234] LustreError: 370742:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [13447.830858] LustreError: 370742:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.134@tcp: cfg command failed: rc = -17 [13447.854354] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.134@tcp [13447.854354] [13753.824300] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13753.835253] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13753.842297] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13753.852807] Lustre: Skipped 2 previous similar messages [13753.862865] Lustre: Skipped 1 previous similar message [13757.433939] Lustre: server umount lustre-MDT0001 complete [13758.947368] LustreError: 358883:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13758.980921] LustreError: 358883:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [13767.973870] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13768.036053] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13768.039760] Lustre: Found index 1 for lustre-MDT0001, updating log [13768.066562] Lustre: 358874:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13768.389826] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13772.711477] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [13773.308992] LustreError: 379213:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13773.318668] LustreError: 379213:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13773.342941] LustreError: 358876:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13773.351654] LustreError: 358876:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13773.451597] LustreError: 379220:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [13773.477994] LustreError: 379220:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.134@tcp: cfg command failed: rc = -114 [13773.487907] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [13773.487907] [13773.807275] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [14096.354370] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [14096.357608] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14096.374655] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14096.400581] Lustre: Skipped 2 previous similar messages [14096.409591] Lustre: Skipped 2 previous similar messages [14099.618707] Lustre: server umount lustre-MDT0000 complete [14101.479196] LustreError: 382831:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14101.508235] LustreError: 382831:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [14103.117873] LustreError: 358866:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786734561 with bad export cookie 9078726492707272358 [14103.119813] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14103.131694] LustreError: 358866:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [14106.605490] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14109.543312] Lustre: server umount lustre-MDT0001 complete [14113.436860] Lustre: server umount lustre-OST0000 complete [14122.442749] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [14131.632262] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [14168.066122] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 15:10:24 (1786734624) [14174.612832] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14174.794252] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [14174.818647] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [14174.883544] Lustre: lustre-MDT0000: new disk, initializing [14174.941924] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [14174.954808] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [14178.503679] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [14187.815473] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14187.966488] Lustre: 389974:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [14188.050639] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [14188.054203] Lustre: Skipped 1 previous similar message [14188.111515] Lustre: lustre-MDT0001: new disk, initializing [14188.192405] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [14188.216459] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [14188.226950] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [14192.481382] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [14201.945054] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [14207.617671] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [14215.054641] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14215.296131] Lustre: lustre-OST0000: new disk, initializing [14215.300751] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [14215.305976] Lustre: 391943:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [14215.371550] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [14220.998255] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [14223.916704] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [14223.924483] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [14224.046469] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [14229.725881] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [14239.218667] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [14239.232840] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14239.270347] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14239.279384] Lustre: Skipped 3 previous similar messages [14240.154691] Lustre: server umount lustre-MDT0000 complete [14244.056220] LustreError: 390769:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786734702 with bad export cookie 9078726492707274318 [14244.066590] LustreError: 390769:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [14244.075335] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14244.076470] LustreError: 390782:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [14244.618202] Lustre: server umount lustre-MDT0001 complete [14259.845599] Lustre: server umount lustre-OST0000 complete [14293.967978] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [14307.232540] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [14343.109091] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 14069 sec ========= 15:13:19 (1786734799) [14345.526509] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 15:13:21 (1786734801) === [14350.097454] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 15:13:26 (1786734806) === [14392.870624] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [14395.292132] Key type lgssc unregistered [14395.511608] LNet: 398313:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [14395.518858] LNetError: 398313:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [14395.534978] LNet: Removed LNI 192.168.202.134@tcp [14396.238864] Key type .llcrypt unregistered [14396.240866] Key type ._llcrypt unregistered