[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-8.fc42 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 467583012 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE2439 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE22D5 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 002295 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE2349 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE23D9 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2411 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe22d5-0xbffe2348] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe22d4] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe2349-0xbffe23d8] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe23d9-0xbffe2410] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2411-0xbffe2438] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524588K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001010] APIC: Switch to symmetric I/O mode setup [ 0.003046] x2apic enabled [ 0.004008] Switched APIC routing to physical x2apic. [ 0.005017] kvm-guest: setup PV IPIs [ 0.008331] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009017] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010013] pid_max: default: 32768 minimum: 301 [ 0.011140] LSM: Security Framework initializing [ 0.012094] Yama: becoming mindful. [ 0.013039] SELinux: Initializing. [ 0.014077] *** VALIDATE selinux *** [ 0.022544] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027210] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028180] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029121] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030113] *** VALIDATE tmpfs *** [ 0.031449] *** VALIDATE proc *** [ 0.032273] *** VALIDATE cgroup *** [ 0.033010] *** VALIDATE cgroup2 *** [ 0.034275] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.036173] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.038004] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.039024] Spectre V2 : User space: Vulnerable [ 0.040009] Speculative Store Bypass: Vulnerable [ 0.042809] debug: unmapping init [mem 0xffffffff9aa59000-0xffffffff9aa60fff] [ 0.044179] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.045657] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.046033] ... version: 2 [ 0.047012] ... bit width: 48 [ 0.047935] ... generic registers: 4 [ 0.048016] ... value mask: 0000ffffffffffff [ 0.049019] ... max period: 00007fffffffffff [ 0.050017] ... fixed-purpose events: 3 [ 0.051026] ... event mask: 000000070000000f [ 0.052318] rcu: Hierarchical SRCU implementation. [ 0.054379] smp: Bringing up secondary CPUs ... [ 0.055592] x86: Booting SMP configuration: [ 0.056026] .... node #0, CPUs: #1 #2 #3 [ 0.059280] smp: Brought up 1 node, 4 CPUs [ 0.061012] smpboot: Max logical packages: 1 [ 0.062018] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.135412] node 0 deferred pages initialised in 72ms [ 0.138653] devtmpfs: initialized [ 0.139237] x86/mm: Memory block size: 128MB [ 0.141505] gcov: version magic: 0x41383552 [ 0.144280] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.147102] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.150306] pinctrl core: initialized pinctrl subsystem [ 0.152177] [ 0.152727] ************************************************************* [ 0.158014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.160012] ** ** [ 0.162012] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.164011] ** ** [ 0.167014] ** This means that this kernel is built to expose internal ** [ 0.169011] ** IOMMU data structures, which may compromise security on ** [ 0.171013] ** your system. ** [ 0.174013] ** ** [ 0.176011] ** If you see this message and you are not debugging the ** [ 0.178013] ** kernel, report this immediately to your vendor! ** [ 0.181014] ** ** [ 0.183013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.185010] ************************************************************* [ 0.186607] NET: Registered protocol family 16 [ 0.188481] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.191058] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.194065] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.198145] cpuidle: using governor menu [ 0.199904] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.201487] PCI: Using configuration type 1 for base access [ 0.203119] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.212050] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.213017] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.214203] cryptd: max_cpu_qlen set to 1000 [ 0.216335] ACPI: Added _OSI(Module Device) [ 0.218012] ACPI: Added _OSI(Processor Device) [ 0.219008] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.220000] ACPI: Added _OSI(Processor Aggregator Device) [ 0.222419] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.228243] ACPI: Interpreter enabled [ 0.229042] ACPI: PM: (supports S0 S3 S4 S5) [ 0.229983] ACPI: Using IOAPIC for interrupt routing [ 0.231078] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.234380] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.241387] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.243087] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.245019] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.247061] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.251045] acpiphp: Slot [2] registered [ 0.252077] acpiphp: Slot [5] registered [ 0.253110] acpiphp: Slot [6] registered [ 0.254087] acpiphp: Slot [7] registered [ 0.255096] acpiphp: Slot [8] registered [ 0.256105] acpiphp: Slot [9] registered [ 0.258121] acpiphp: Slot [10] registered [ 0.259090] acpiphp: Slot [3] registered [ 0.259971] acpiphp: Slot [4] registered [ 0.260066] acpiphp: Slot [11] registered [ 0.262101] acpiphp: Slot [12] registered [ 0.263108] acpiphp: Slot [13] registered [ 0.264100] acpiphp: Slot [14] registered [ 0.266088] acpiphp: Slot [15] registered [ 0.267089] acpiphp: Slot [16] registered [ 0.268105] acpiphp: Slot [17] registered [ 0.269086] acpiphp: Slot [18] registered [ 0.271119] acpiphp: Slot [19] registered [ 0.272093] acpiphp: Slot [20] registered [ 0.273066] acpiphp: Slot [21] registered [ 0.274100] acpiphp: Slot [22] registered [ 0.276069] acpiphp: Slot [23] registered [ 0.277061] acpiphp: Slot [24] registered [ 0.278076] acpiphp: Slot [25] registered [ 0.279060] acpiphp: Slot [26] registered [ 0.280066] acpiphp: Slot [27] registered [ 0.281079] acpiphp: Slot [28] registered [ 0.282070] acpiphp: Slot [29] registered [ 0.283076] acpiphp: Slot [30] registered [ 0.284068] acpiphp: Slot [31] registered [ 0.285043] PCI host bridge to bus 0000:00 [ 0.286013] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.288020] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.290014] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.291015] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.293016] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.295017] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.296119] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.297860] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.299933] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.307011] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.310101] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.312013] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.314014] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.316013] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.317460] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.319582] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.321030] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.323603] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.327014] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.336016] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.339014] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.343143] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.349021] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.353016] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.365016] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.378213] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.385015] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.391017] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.410016] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.417570] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.422015] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.429026] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.439016] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.446808] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.450015] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.455018] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.465016] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.472522] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.477012] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.482017] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.494015] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.501035] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.506015] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.510015] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.521015] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.534622] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.536314] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.538218] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.540375] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.542155] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.546079] iommu: Default domain type: Passthrough [ 0.547423] SCSI subsystem initialized [ 0.549261] ACPI: bus type USB registered [ 0.550096] usbcore: registered new interface driver usbfs [ 0.552063] usbcore: registered new interface driver hub [ 0.554059] usbcore: registered new device driver usb [ 0.555109] pps_core: LinuxPPS API ver. 1 registered [ 0.556007] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.558049] PTP clock support registered [ 0.559075] EDAC MC: Ver: 3.0.0 [ 0.561128] PCI: Using ACPI for IRQ routing [ 0.563870] NetLabel: Initializing [ 0.565010] NetLabel: domain hash size = 128 [ 0.567011] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.570085] NetLabel: unlabeled traffic allowed by default [ 0.572143] vgaarb: loaded [ 0.573286] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.574011] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.579768] clocksource: Switched to clocksource kvm-clock [ 0.686265] VFS: Disk quotas dquot_6.6.0 [ 0.687784] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.690163] *** VALIDATE ramfs *** [ 0.691487] *** VALIDATE hugetlbfs *** [ 0.692963] pnp: PnP ACPI init [ 0.695199] pnp: PnP ACPI: found 6 devices [ 0.710418] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.713649] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.715790] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.717958] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.720576] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.723190] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.725984] NET: Registered protocol family 2 [ 0.728722] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.733889] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.737786] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.743362] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.746928] TCP: Hash tables configured (established 65536 bind 65536) [ 0.750154] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.753446] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.756557] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.759843] NET: Registered protocol family 1 [ 0.762601] RPC: Registered named UNIX socket transport module. [ 0.765275] RPC: Registered udp transport module. [ 0.767028] RPC: Registered tcp transport module. [ 0.768478] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.771193] NET: Registered protocol family 44 [ 0.772801] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.774941] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.777028] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.779225] PCI: CLS 0 bytes, default 64 [ 0.781533] Unpacking initramfs... [ 2.173404] debug: unmapping init [mem 0xffff8a87fcc54000-0xffff8a87fffbffff] [ 2.177495] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.179915] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.183106] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.658396] Initialise system trusted keyrings [ 2.659814] Key type blacklist registered [ 2.661596] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.671682] zbud: loaded [ 2.675251] *** VALIDATE nfs *** [ 2.676300] *** VALIDATE nfs4 *** [ 2.677585] pstore: using deflate compression [ 2.681314] Platform Keyring initialized [ 2.801269] NET: Registered protocol family 38 [ 2.803166] Key type asymmetric registered [ 2.805115] Asymmetric key parser 'x509' registered [ 2.806955] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.810292] io scheduler mq-deadline registered [ 2.812157] io scheduler kyber registered [ 2.813795] io scheduler bfq registered [ 2.815650] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.818344] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.821182] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.824745] ACPI: Power Button [PWRF] [ 2.830619] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.837217] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.850693] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 2.859450] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 2.876049] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.905342] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.934886] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 2.940473] Non-volatile memory driver v1.3 [ 2.942370] Linux agpgart interface v0.103 [ 2.982865] virtio_blk virtio1: [vda] 141880 512-byte logical blocks (72.6 MB/69.3 MiB) [ 2.986293] vda: detected capacity change from 0 to 72642560 [ 3.001181] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.004475] vdb: detected capacity change from 0 to 1073741824 [ 3.019180] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.022174] vdc: detected capacity change from 0 to 2621440000 [ 3.041539] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.044987] vdd: detected capacity change from 0 to 2621440000 [ 3.064429] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.067304] vde: detected capacity change from 0 to 4294967296 [ 3.083492] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.087083] vdf: detected capacity change from 0 to 4294967296 [ 3.096728] libphy: Fixed MDIO Bus: probed [ 3.106866] usbcore: registered new interface driver usbserial_generic [ 3.110164] usbserial: USB Serial support registered for generic [ 3.112891] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.117892] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.120328] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.123570] mousedev: PS/2 mouse device common for all mice [ 3.127169] rtc_cmos 00:05: RTC can wake from S4 [ 3.129913] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.130574] rtc_cmos 00:05: registered as rtc0 [ 3.135109] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.138120] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.138739] intel_pstate: CPU model not supported [ 3.145474] hid: raw HID events driver (C) Jiri Kosina [ 3.147700] usbcore: registered new interface driver usbhid [ 3.148269] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.149640] usbhid: USB HID core driver [ 3.154281] drop_monitor: Initializing network drop monitor service [ 3.156615] Initializing XFRM netlink socket [ 3.158707] NET: Registered protocol family 10 [ 3.161535] Segment Routing with IPv6 [ 3.162922] NET: Registered protocol family 17 [ 3.165228] mpls_gso: MPLS GSO support [ 3.170540] RAS: Correctable Errors collector initialized. [ 3.172643] AVX version of gcm_enc/dec engaged. [ 3.174232] AES CTR mode by8 optimization enabled [ 3.256163] sched_clock: Marking stable (3256114503, 0)->(4161745252, -905630749) [ 3.260073] registered taskstats version 1 [ 3.262261] Loading compiled-in X.509 certificates [ 3.264635] zswap: loaded using pool lzo/zbud [ 3.294073] Key type big_key registered [ 3.305335] Key type encrypted registered [ 3.306871] ima: No TPM chip found, activating TPM-bypass! [ 3.309533] ima: Allocated hash algorithm: sha1 [ 3.311410] ima: No architecture policies found [ 3.314922] evm: Initialising EVM extended attributes: [ 3.316912] evm: security.selinux [ 3.318255] evm: security.ima [ 3.319301] evm: security.capability [ 3.320528] evm: HMAC attrs: 0x1 [ 3.322878] rtc_cmos 00:05: setting system clock to 2026-05-19 14:43:58 UTC (1779201838) [ 3.329154] debug: unmapping init [mem 0xffffffff9ba03000-0xffffffff9bbfffff] [ 3.332326] debug: unmapping init [mem 0xffffffff9a782000-0xffffffff9aa58fff] [ 3.340207] Write protecting the kernel read-only data: 28672k [ 3.343675] debug: unmapping init [mem 0xffffffff98e03000-0xffffffff98ffffff] [ 3.346697] debug: unmapping init [mem 0xffffffff99714000-0xffffffff997fffff] [ 3.380444] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.389398] systemd[1]: Detected virtualization kvm. [ 3.391768] systemd[1]: Detected architecture x86-64. [ 3.393844] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.416090] systemd[1]: No hostname configured. [ 3.417792] systemd[1]: Set hostname to . [ 3.419972] random: systemd: uninitialized urandom read (16 bytes read) [ 3.422857] systemd[1]: Initializing machine ID from random generator. [ 3.578413] random: systemd: uninitialized urandom read (16 bytes read) [ 3.581801] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 3.586099] random: systemd: uninitialized urandom read (16 bytes read) [ 3.593231] systemd[1]: Starting Create list of required static device nodes for the current kernel... Starting Create list of required st…ce nodes for the current kernel... [ 3.601292] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Local File Systems. [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Swap. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Listening on udev Control Socket. [ OK ] Reached target Timers. Starting Create Volatile Files and Directories... [ OK ] Reached target Slices. [ OK ] Reached target Sockets. Starting Apply Kernel Variables... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Initrd Root Device. Starting Setup Virtual Console... [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.209702] device-mapper: uevent: version 1.0.3 [ 4.212509] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 4.889689] virtio_net virtio0 ens2: renamed from eth0 [ 4.910756] random: fast init done [ 4.953197] scsi host0: ata_piix [ 4.975177] scsi host1: ata_piix [ 4.977594] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 4.979964] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.286820] dracut-initqueue[592]: RTNETLINK answers: File exists [ 9.824689] random: crng init done [ 9.826228] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.175437] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Swap. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Slices. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.287899] printk: systemd: 26 output lines suppressed due to ratelimiting [ 11.566718] SELinux: Disabled at runtime. [ 11.632055] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.640460] systemd[1]: Detected virtualization kvm. [ 11.642541] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.126840] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.129737] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.133994] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.139405] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.144604] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.151435] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.155708] systemd[1]: Created slice system-serial\x2dgetty.slice. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Listening on udev Kernel Socket. Starting Apply Kernel Variables... [ OK ] Created slice User and Session Slice. Starting Create list of required st…ce nodes for the current kernel... Mounting Kernel Debug File System... Mounting Huge Pages File System... [ OK ] Reached target Slices. [ OK ] Created slice system-sshd\x2dkeygen.slice. Starting Remount Root and Kernel File Systems... [ OK ] Listening on RPCbind Server Activation Socket. Activating swap /dev/disk/by-label/SWAP... [ OK ] Created slice system-getty.slice. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Reached target RPC Port Mapper. [ OK ] Reached target rpc_pipefs.target. [ OK ] Listening on Process Core Du[ 12.277150] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS mp Socket. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Stopped target Initrd File Systems. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... Mounting POSIX Message Queue File System... [ OK ] Started Journal Service. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted Huge Pages File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ OK ] Started udev Coldplug all Devices. [ 12.610735] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 12.981994] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.033587] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.112458] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.131193] EDAC sbridge: Ver: 1.1.2 [ 14.842906] Key type dns_resolver registered [ 15.167279] NFS: Registering the id_resolver key type [ 15.168639] Key type id_resolver registered [ 15.169965] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... Starting Login Service... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. [ OK ] Started Login Service. [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... [ OK ] Started OpenSSH server daemon. Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. [ OK ] Started Authorization Manager. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg229-server login: [ 41.718371] libcfs: loading out-of-tree module taints kernel. [ 41.735567] Key type ._llcrypt registered [ 41.737303] Key type .llcrypt registered [ 41.781736] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_hostid [ 61.192359] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 63.595500] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 63.607532] alg: No test for adler32 (adler32-zlib) [ 65.469801] Lustre: Lustre: Build Version: 2.17.53_25_ge261312 [ 66.670648] LNet: Added LNI 192.168.202.129@tcp [8/256/0/180] [ 68.639544] Key type lgssc registered [ 70.150414] hrtimer: interrupt took 7406286 ns [ 71.875624] Lustre: Echo OBD driver; http://www.lustre.org/ [ 98.808967] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 155.274413] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 175.320199] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 175.372379] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 176.784424] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 176.844667] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 177.013991] Lustre: lustre-MDT0000: new disk, initializing [ 177.160735] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 177.195287] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 182.595871] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 199.214260] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 199.298799] Lustre: 6567:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 199.328966] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 199.333835] Lustre: Skipped 1 previous similar message [ 199.449717] Lustre: lustre-MDT0001: new disk, initializing [ 199.547942] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 199.578741] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 199.606596] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 205.323780] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 211.999103] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 225.874625] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 226.481651] Lustre: lustre-OST0000: new disk, initializing [ 226.490700] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 226.495571] Lustre: 8474:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 226.623419] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 233.537916] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 233.540880] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 233.622096] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 233.927755] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 251.728829] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 251.873618] Lustre: lustre-OST0001: new disk, initializing [ 251.876979] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 251.882541] Lustre: 9532:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 251.964702] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 258.080947] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 258.085384] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 258.283051] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 258.900753] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 271.839885] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 276.679929] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 283.784926] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing check_logdir /tmp/testlogs/ [ 289.604529] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing yml_node [ 295.966728] Lustre: DEBUG MARKER: Client: 2.17.53.25 [ 299.046125] Lustre: DEBUG MARKER: MDS: 2.17.53.25 [ 301.984918] Lustre: DEBUG MARKER: OSS: 2.17.53.25 [ 303.683941] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Tue May 19 10:48:57 EDT 2026 [ 324.237914] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 325.903328] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 343.008361] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 343.018702] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 343.046221] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 345.568214] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 345.570734] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 345.583742] Lustre: Skipped 2 previous similar messages [ 349.158768] Lustre: server umount lustre-MDT0000 complete [ 353.260176] LustreError: 6562:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779202188 with bad export cookie 621376208814234744 [ 353.271689] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 355.807850] LustreError: 6577:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 355.808027] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 355.808587] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 355.808591] Lustre: Skipped 3 previous similar messages [ 355.830834] LustreError: 6577:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 355.884538] Lustre: Skipped 1 previous similar message [ 358.815920] LustreError: 6575:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 358.816715] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 358.858320] Lustre: Skipped 2 previous similar messages [ 359.754330] Lustre: server umount lustre-MDT0001 complete [ 364.900397] Lustre: server umount lustre-OST0000 complete [ 369.840574] Lustre: server umount lustre-OST0001 complete [ 377.555332] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_hostid [ 388.964652] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 423.864672] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 424.077053] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 424.107655] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 424.185366] Lustre: lustre-MDT0000: new disk, initializing [ 424.326279] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 424.356403] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 428.119295] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 439.409702] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 439.554995] Lustre: 15062:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 439.639146] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 439.646558] Lustre: Skipped 1 previous similar message [ 439.765679] Lustre: lustre-MDT0001: new disk, initializing [ 440.007463] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 440.105713] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 440.123841] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 445.539469] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 455.785092] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 463.288306] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 472.699809] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 473.111750] Lustre: lustre-OST0000: new disk, initializing [ 473.118867] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 473.124917] Lustre: 16808:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 473.227649] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 475.135932] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 475.161626] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 475.306230] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 481.198568] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 490.313562] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 495.995557] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 496.186331] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 501.066550] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 501.342277] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 505.838351] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 505.850314] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 505.858638] Lustre: Skipped 1 previous similar message [ 505.861847] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 505.869265] Lustre: Skipped 1 previous similar message [ 509.788483] Lustre: server umount lustre-OST0000 complete [ 516.075643] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 516.095042] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 520.538361] Lustre: server umount lustre-MDT0000 complete [ 521.187770] LustreError: 15069:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 521.210397] LustreError: 15069:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 524.280633] LustreError: 15840:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779202359 with bad export cookie 621376208814236270 [ 524.284179] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 524.297073] LustreError: 15840:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 524.627822] Lustre: server umount lustre-MDT0001 complete [ 529.988934] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 10:52:43 (1779202363) [ 536.935883] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 537.432804] LustreError: 18924:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 537.524320] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 542.703434] LustreError: 18925:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 542.803981] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 552.957424] LustreError: 18924:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 553.006441] LustreError: 18924:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 553.591661] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 559.534377] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 564.943583] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 569.200673] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 576.626968] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 584.514335] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 591.274815] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 595.026791] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 603.108726] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 603.124358] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 603.127528] Lustre: Skipped 2 previous similar messages [ 603.146295] Lustre: Skipped 3 previous similar messages [ 608.237480] LustreError: 18924:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 608.265391] LustreError: 18924:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 608.441239] Lustre: server umount lustre-MDT0000 complete [ 612.561142] LustreError: 18906:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779202447 with bad export cookie 621376208814237159 [ 612.568200] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 612.576025] LustreError: 18906:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 612.880245] Lustre: server umount lustre-MDT0001 complete [ 627.522541] Lustre: server umount lustre-OST0000 complete [ 651.238401] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 661.936462] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 662.575774] LustreError: 22957:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 662.663362] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 662.670670] Lustre: Skipped 2 previous similar messages [ 667.455415] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 675.728504] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 681.165711] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 684.072949] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 690.786269] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 698.692205] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 709.030104] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 709.328387] Lustre: lustre-OST0001: new disk, initializing [ 709.333961] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 709.344202] Lustre: 25225:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 710.686305] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 710.697512] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 710.790193] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 715.798212] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 725.536294] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 729.329383] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 742.881461] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 742.896159] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 742.912975] Lustre: Skipped 1 previous similar message [ 742.938769] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 749.010359] Lustre: server umount lustre-MDT0000 complete [ 751.586728] LustreError: 25235:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 751.620302] LustreError: 25235:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 753.212959] LustreError: 22937:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779202588 with bad export cookie 621376208814238006 [ 753.221110] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 753.227534] LustreError: 22937:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 753.636164] Lustre: server umount lustre-MDT0001 complete [ 768.586175] Lustre: server umount lustre-OST0000 complete [ 772.065465] Lustre: 3654:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779202591/real 1779202591] req@ffff8a886a5a5500 x1865628412796672/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779202607 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 773.254588] Lustre: server umount lustre-OST0001 complete [ 792.417776] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 804.679273] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 805.452964] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 805.460271] Lustre: Skipped 3 previous similar messages [ 810.028566] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 820.824122] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 826.194997] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 829.254408] Lustre: 29332:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 836.055201] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 842.053823] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 850.480470] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 857.117849] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 864.459202] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 868.097543] Lustre: 31175:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 876.514416] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 876.519158] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 876.535845] Lustre: Skipped 7 previous similar messages [ 876.554377] Lustre: Skipped 7 previous similar messages [ 880.371302] Lustre: server umount lustre-MDT0000 complete [ 881.643808] LustreError: 29402:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 881.666299] LustreError: 29402:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 8 previous similar messages [ 884.160079] LustreError: 28201:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779202719 with bad export cookie 621376208814239294 [ 884.179372] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 884.180892] LustreError: 28201:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 884.493291] Lustre: server umount lustre-MDT0001 complete [ 898.807231] Lustre: server umount lustre-OST0000 complete [ 913.043606] Lustre: server umount lustre-OST0001 complete [ 921.257786] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 10:59:14 (1779202754) [ 951.153098] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 963.742735] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 968.641297] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 976.844760] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 982.575947] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 985.947661] Lustre: 35086:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 993.151610] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1000.921076] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 1010.046095] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1016.381167] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 1024.509780] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1028.754275] Lustre: 36944:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1036.262542] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1036.276740] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1036.287817] Lustre: Skipped 3 previous similar messages [ 1036.309060] Lustre: Skipped 3 previous similar messages [ 1040.658091] Lustre: server umount lustre-MDT0000 complete [ 1044.494795] LustreError: 33953:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779202879 with bad export cookie 621376208814240407 [ 1044.499757] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1044.516786] LustreError: 33953:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 1044.818280] Lustre: server umount lustre-MDT0001 complete [ 1058.186787] Lustre: server umount lustre-OST0000 complete [ 1062.377194] Lustre: 3653:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779202881/real 1779202881] req@ffff8a88509f7800 x1865628412893440/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779202897 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1062.846161] Lustre: server umount lustre-OST0001 complete [ 1071.695692] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 11:01:45 (1779202905) [ 1089.262269] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 1101.010691] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1101.763560] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1101.777993] Lustre: Skipped 7 previous similar messages [ 1107.285556] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 1116.862434] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1122.310774] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 1126.277994] Lustre: 40193:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1134.756084] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1141.518287] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 1142.327948] LustreError: 40550:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1142.346520] LustreError: 40550:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 15 previous similar messages [ 1151.884634] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1159.244393] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 1168.990551] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1173.526292] Lustre: 42038:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1174.517116] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1174.522278] Lustre: Skipped 1 previous similar message [ 1178.102612] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1178.132091] Lustre: Skipped 3 previous similar messages [ 1183.947816] Lustre: server umount lustre-MDT0000 complete [ 1189.473483] LustreError: 39060:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779203024 with bad export cookie 621376208814241506 [ 1189.486550] LustreError: 39060:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1189.488880] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1196.031702] Lustre: server umount lustre-MDT0001 complete [ 1205.366498] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1213.919825] LustreError: 3650:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff8a8850785880 x1865628412949632/t0(0) o250->MGC192.168.202.129@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1219.233665] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 1224.696131] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1228.740043] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1234.428422] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1234.444167] Lustre: Skipped 1 previous similar message [ 1234.892384] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 1241.728672] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1243.522390] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1247.706700] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1249.320232] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1262.289969] Lustre: server umount lustre-MDT0000 complete [ 1266.134538] LustreError: 39060:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779203101 with bad export cookie 621376208814242605 [ 1266.136724] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1266.150358] LustreError: 39060:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1266.454987] Lustre: server umount lustre-MDT0001 complete [ 1281.397516] Lustre: server umount lustre-OST0000 complete [ 1296.505684] Lustre: server umount lustre-OST0001 complete [ 1306.510546] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 11:05:40 (1779203140) [ 1326.623056] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 1338.796200] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1345.042238] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 1354.970341] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1360.410737] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 1363.774341] Lustre: 47535:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1370.634872] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1378.250484] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 1387.317992] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1395.556478] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 1404.133320] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1409.989192] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1426.912052] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1426.919027] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1426.926847] Lustre: Skipped 19 previous similar messages [ 1430.485861] Lustre: server umount lustre-MDT0000 complete [ 1434.775351] LustreError: 48711:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779203269 with bad export cookie 621376208814243431 [ 1434.778686] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1434.787611] LustreError: 48711:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1435.187806] Lustre: server umount lustre-MDT0001 complete [ 1449.664774] Lustre: server umount lustre-OST0000 complete [ 1463.918815] Lustre: server umount lustre-OST0001 complete [ 1473.530042] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 11:08:27 (1779203307) [ 1475.733821] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1477.235818] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 11:08:31 (1779203311) [ 1487.173341] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_hostid [ 1494.861188] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 1515.798328] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1516.069252] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1516.186640] Lustre: lustre-MDT0000: new disk, initializing [ 1516.368567] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1520.696640] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 1528.449053] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1535.076726] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1535.428632] Lustre: lustre-OST0000: new disk, initializing [ 1535.438123] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1535.443791] Lustre: Skipped 1 previous similar message [ 1535.455849] Lustre: 53450:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1536.905050] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1536.924306] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1536.993726] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1541.467310] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 1550.223528] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1552.577924] Lustre: DEBUG MARKER: create test files [ 1588.193078] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1588.199300] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1588.213200] Lustre: Skipped 11 previous similar messages [ 1593.297330] Lustre: server umount lustre-OST0000 complete [ 1597.989292] Lustre: server umount lustre-MDT0000 complete [ 1610.723353] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing unload_modules_local [ 1614.144481] Key type lgssc unregistered [ 1614.522553] LNet: 54956:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1614.531349] LNetError: 54956:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1614.548512] LNet: Removed LNI 192.168.202.129@tcp [ 1615.722397] Key type .llcrypt unregistered [ 1615.724733] Key type ._llcrypt unregistered [ 1647.232203] Key type ._llcrypt registered [ 1647.244413] Key type .llcrypt registered [ 1647.448794] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 1649.007400] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1649.220279] alg: No test for adler32 (adler32-zlib) [ 1650.489793] Lustre: Lustre: Build Version: 2.17.53_25_ge261312 [ 1650.880528] LNet: Added LNI 192.168.202.129@tcp [8/256/0/180] [ 1652.655397] Key type lgssc registered [ 1654.459267] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1665.854359] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1665.940047] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1667.765144] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1672.114319] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 1678.291110] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1685.431055] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1685.890574] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1692.019276] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1692.139372] Lustre: 57630:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1692.154537] Lustre: 57630:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1692.160596] Lustre: 57630:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1692.172563] Lustre: 57630:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1692.178759] Lustre: 57630:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1692.188903] Lustre: 57630:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1693.625984] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 1701.132607] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1708.426772] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1715.964461] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1751.135560] Lustre: server umount lustre-OST0000 complete [ 1766.596631] Lustre: server umount lustre-MDT0000 complete [ 1786.691318] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing unload_modules_local [ 1792.297518] Key type lgssc unregistered [ 1792.726794] LNet: 59155:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1792.739248] LNetError: 59155:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1792.779612] LNet: Removed LNI 192.168.202.129@tcp [ 1794.074921] Key type .llcrypt unregistered [ 1794.079828] Key type ._llcrypt unregistered [ 1832.930250] Key type ._llcrypt registered [ 1832.932552] Key type .llcrypt registered [ 1833.114916] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 1835.708437] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1835.771401] alg: No test for adler32 (adler32-zlib) [ 1837.050600] Lustre: Lustre: Build Version: 2.17.53_25_ge261312 [ 1837.465437] LNet: Added LNI 192.168.202.129@tcp [8/256/0/180] [ 1839.314935] Key type lgssc registered [ 1841.034953] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1852.269836] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1852.322393] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1853.951788] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1860.066845] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 1866.665895] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1874.516510] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1874.877844] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1879.042117] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1879.133050] Lustre: 61917:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 514 < left 544, rollback = 9 [ 1879.141130] Lustre: 61917:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/1, destroy: 0/0/0 [ 1879.149338] Lustre: 61917:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1879.155198] Lustre: 61917:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1879.167764] Lustre: 61917:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1879.179343] Lustre: 61917:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1882.205373] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 1888.758466] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1894.873656] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1909.269919] Lustre: server umount lustre-OST0000 complete [ 1924.804709] Lustre: server umount lustre-MDT0000 complete [ 1937.437866] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing unload_modules_local [ 1940.404624] Key type lgssc unregistered [ 1940.852461] LNet: 63356:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1940.866922] LNetError: 63356:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1940.879171] LNet: Removed LNI 192.168.202.129@tcp [ 1941.611404] Key type .llcrypt unregistered [ 1941.613808] Key type ._llcrypt unregistered [ 1963.373459] Key type ._llcrypt registered [ 1963.376839] Key type .llcrypt registered [ 1963.510957] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_hostid [ 1981.918614] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 1983.351213] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1983.399752] alg: No test for adler32 (adler32-zlib) [ 1984.458810] Lustre: Lustre: Build Version: 2.17.53_25_ge261312 [ 1984.693222] LNet: Added LNI 192.168.202.129@tcp [8/256/0/180] [ 1986.391718] Key type lgssc registered [ 1987.451173] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2029.035840] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2029.077777] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2030.510136] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2030.580820] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2030.727345] Lustre: lustre-MDT0000: new disk, initializing [ 2030.810327] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2030.831614] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2035.803462] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 2048.610925] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2048.790219] Lustre: 66535:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2048.857528] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2048.864052] Lustre: Skipped 1 previous similar message [ 2048.936603] Lustre: lustre-MDT0001: new disk, initializing [ 2049.119161] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2049.207750] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2049.235711] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2053.959911] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 2064.459667] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2069.501084] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2079.049618] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2079.485173] Lustre: lustre-OST0000: new disk, initializing [ 2079.493316] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2079.503751] Lustre: 68277:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2079.625297] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2087.535646] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 2088.037435] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2088.047971] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2088.171521] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2095.951643] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2100.342631] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2100.526778] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2105.415483] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2105.708796] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2108.899218] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2108.908244] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2108.917972] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2108.933741] Lustre: Skipped 1 previous similar message [ 2108.951724] Lustre: Skipped 1 previous similar message [ 2114.036470] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2114.039192] Lustre: Skipped 1 previous similar message [ 2119.141042] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2119.143805] Lustre: Skipped 1 previous similar message [ 2122.207119] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2122.449197] Lustre: server umount lustre-OST0000 complete [ 2129.376700] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2129.398414] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2129.406398] Lustre: Skipped 1 previous similar message [ 2129.419383] Lustre: Skipped 1 previous similar message [ 2132.614690] Lustre: server umount lustre-MDT0000 complete [ 2136.294667] LustreError: 66528:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779203971 with bad export cookie 6523751853971947213 [ 2136.306602] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2136.307524] LustreError: 66528:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2136.697474] Lustre: server umount lustre-MDT0001 complete [ 2144.818833] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 11:19:38 (1779203978) [ 2157.310874] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 2198.316528] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_hostid [ 2207.266357] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 2238.250786] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2238.424227] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2238.447927] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2238.531401] Lustre: lustre-MDT0000: new disk, initializing [ 2238.599752] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2238.614493] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2242.535898] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 2253.824313] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2253.900730] Lustre: 74115:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2253.988069] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2253.998329] Lustre: Skipped 1 previous similar message [ 2254.087860] Lustre: lustre-MDT0001: new disk, initializing [ 2254.199610] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2254.250185] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2254.277986] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2260.829495] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 2270.996819] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2275.054261] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2282.236560] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2282.594348] Lustre: lustre-OST0000: new disk, initializing [ 2282.600395] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2282.614390] Lustre: 75858:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2282.740929] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2288.880519] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 2291.853624] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2291.882719] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2292.142718] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2296.332948] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2302.830711] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2303.288116] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2309.773818] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2310.049422] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2317.799319] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2317.812776] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2317.825103] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2317.838264] LustreError: Skipped 2 previous similar messages [ 2327.007174] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2327.249844] Lustre: server umount lustre-OST0000 complete [ 2333.154721] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2333.157680] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2333.173603] Lustre: Skipped 2 previous similar messages [ 2333.178090] Lustre: Skipped 3 previous similar messages [ 2339.116414] Lustre: server umount lustre-MDT0000 complete [ 2342.823321] LustreError: 74107:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779204177 with bad export cookie 6523751853971948095 [ 2342.827279] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2342.859542] LustreError: 74107:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2343.555503] Lustre: server umount lustre-MDT0001 complete [ 2352.497427] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 11:23:06 (1779204186) [ 2359.452952] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2359.929848] LustreError: 78124:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2359.949832] LustreError: 78124:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2360.029196] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2365.053922] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 2365.411516] LustreError: 78124:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2365.432563] LustreError: 78124:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2370.530689] LustreError: 78125:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2373.989714] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2379.260948] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 2384.248475] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2387.965582] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2396.567135] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2396.853447] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2396.864762] Lustre: Skipped 1 previous similar message [ 2402.748838] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 2408.645856] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2411.982425] Lustre: 80456:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 43f8e7e4-3e38-4a91-8fd5-873a68746ed8 at adminstrative request [ 2415.585212] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2415.607408] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2425.853452] Lustre: MGS: Client 43f8e7e4-3e38-4a91-8fd5-873a68746ed8 (at 0@lo) reconnecting [ 2425.871420] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x5a8905bf32cb45b1 to 0x5a8905bf32cb4851 [ 2425.895888] Lustre: MGC192.168.202.129@tcp: Connection restored to 0@lo (at 0@lo) [ 2453.840884] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2454.024731] Lustre: lustre-OST0001: new disk, initializing [ 2454.028286] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2454.033471] Lustre: 80850:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2460.856893] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2460.892107] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2461.045535] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2463.225336] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 2471.072194] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2476.512086] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2476.515108] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2476.518381] LustreError: Skipped 1 previous similar message [ 2476.536850] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2476.540866] Lustre: Skipped 1 previous similar message [ 2476.551596] Lustre: Skipped 4 previous similar messages [ 2480.560557] Lustre: server umount lustre-OST0001 complete [ 2481.641102] LustreError: 79719:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2481.667355] LustreError: 79719:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2486.772741] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2486.783178] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2486.786322] LustreError: 79719:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2486.802869] Lustre: Skipped 1 previous similar message [ 2492.239534] Lustre: server umount lustre-OST0000 complete [ 2504.779897] Lustre: server umount lustre-MDT0000 complete [ 2507.233470] LustreError: 78125:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2507.268348] LustreError: 78125:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 2509.493069] LustreError: 81591:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779204344 with bad export cookie 6523751853971949649 [ 2509.495841] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2509.497440] LustreError: 81591:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2510.137973] Lustre: server umount lustre-MDT0001 complete [ 2522.247261] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 11:25:55 (1779204355) [ 2524.436845] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2527.059335] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 11:26:00 (1779204360) [ 2528.984702] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2531.486442] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 11:26:04 (1779204364) [ 2534.098144] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2536.718216] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 11:26:09 (1779204369) [ 2538.972654] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2541.188620] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 11:26:14 (1779204374) [ 2542.940825] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2550.488747] loop: module loaded [ 2550.519972] loop0: detected capacity change from 0 to 4294967296 [ 2550.585199] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2552.922931] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2557.198730] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2562.670775] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2591.841272] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2591.883990] LustreError: 84845:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2591.889830] LustreError: 84845:0:(obd_mount.c:250:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2591.898431] LustreError: 84845:0:(tgt_mount.c:2534:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2591.916570] LustreError: 84845:0:(super25.c:184:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2620.211407] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_hostid [ 2634.801377] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 2681.542753] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2681.949338] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2682.023399] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2682.177824] Lustre: lustre-MDT0000: new disk, initializing [ 2682.370747] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2682.382947] Lustre: Skipped 1 previous similar message [ 2682.403577] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2687.925738] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 2703.430699] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2703.630449] Lustre: 87689:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2704.295244] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2710.023294] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 2720.423226] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2725.919712] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2733.930851] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2734.492594] Lustre: 89432:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2741.877259] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2742.115622] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2743.601235] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 2752.765726] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2759.535758] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2759.790322] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2764.183269] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2764.403303] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2767.841227] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2767.845938] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2767.857252] LustreError: Skipped 1 previous similar message [ 2767.857979] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2767.857984] Lustre: Skipped 4 previous similar messages [ 2767.897778] Lustre: Skipped 3 previous similar messages [ 2773.644256] Lustre: server umount lustre-OST0000 complete [ 2784.586752] Lustre: server umount lustre-MDT0000 complete [ 2788.323172] LustreError: 87700:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2788.353827] LustreError: 87700:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2788.430734] LustreError: 87680:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779204623 with bad export cookie 6523751853971950265 [ 2788.435954] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2788.440483] LustreError: 87680:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2789.075424] Lustre: server umount lustre-MDT0001 complete [ 2797.798854] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 11:30:31 (1779204631) [ 2804.639280] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2809.699638] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 2818.769380] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2819.097163] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2819.101292] Lustre: Skipped 3 previous similar messages [ 2823.751751] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 2829.298903] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2833.306688] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2839.663701] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2846.305804] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 2852.240952] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2857.338703] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2857.570808] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2862.298311] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2862.564522] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2869.886020] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2870.179532] Lustre: lustre-OST0001: new disk, initializing [ 2870.184390] Lustre: Skipped 2 previous similar messages [ 2870.190580] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2870.198343] Lustre: Skipped 3 previous similar messages [ 2870.211962] Lustre: 94695:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2877.355734] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 2879.088214] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2879.101380] Lustre: Skipped 2 previous similar messages [ 2879.113515] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2879.245658] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2885.563041] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2890.373333] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2890.572190] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2897.067912] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2897.501406] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2918.835785] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2947.174406] Lustre: Failing over lustre-MDT0000 [ 2947.777613] Lustre: server umount lustre-MDT0000 complete [ 2951.144454] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2951.152174] LustreError: 91696:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2951.153909] Lustre: Skipped 4 previous similar messages [ 2951.178862] LustreError: 91696:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 2968.042407] Lustre: 64724:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779204786/real 1779204786] req@0000000064d4dfb3 x1865630423994240/t0(0) o400->MGC192.168.202.129@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1779204802 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2968.067284] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2971.341231] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2971.344320] LDISKFS-fs (dm-0): recovery complete [ 2971.361501] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2978.386518] Lustre: lustre-MDT0000: Not available for connect from 192.168.202.29@tcp (not set up) [ 2978.398350] Lustre: Skipped 8 previous similar messages [ 2978.680505] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2982.880416] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 2983.485445] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2983.486197] Lustre: 91696:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2983.923327] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2983.987361] Lustre: 97393:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2984.001437] Lustre: 97393:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 6 previous similar messages [ 2984.007793] LustreError: 97393:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2984.338783] LustreError: 97393:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2984.667074] LustreError: 97393:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2984.675451] LustreError: 97393:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 1 previous similar message [ 2984.991254] LustreError: 97393:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2985.003932] LustreError: 97393:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 1 previous similar message [ 2985.026043] Lustre: 97393:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2985.034477] Lustre: 97393:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 2 previous similar messages [ 2985.697170] LustreError: 97393:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2985.709804] LustreError: 97393:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 2 previous similar messages [ 2986.031132] LustreError: 97393:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2986.039438] LustreError: 97393:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 2 previous similar messages [ 2987.328498] Lustre: 97393:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2987.339638] Lustre: 97393:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 6 previous similar messages [ 2987.979088] LustreError: 97393:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2987.990674] LustreError: 97393:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 6 previous similar messages [ 2988.311180] LustreError: 97393:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2988.322651] LustreError: 97393:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 6 previous similar messages [ 2991.488051] Lustre: 97393:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2991.505733] Lustre: 97393:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 12 previous similar messages [ 2992.150806] LustreError: 97393:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2992.164143] LustreError: 97393:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 12 previous similar messages [ 2992.479139] LustreError: 97393:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2992.486787] LustreError: 97393:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 12 previous similar messages [ 2999.770001] Lustre: 97393:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2999.792288] Lustre: 97393:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 25 previous similar messages [ 3000.448289] LustreError: 97393:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3000.463953] LustreError: 97393:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 25 previous similar messages [ 3000.791716] LustreError: 97393:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3000.805269] LustreError: 97393:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 25 previous similar messages [ 3015.931509] Lustre: 97393:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3015.950338] Lustre: 97393:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 49 previous similar messages [ 3016.608613] LustreError: 97393:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3016.630638] LustreError: 97393:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 49 previous similar messages [ 3016.960194] LustreError: 97393:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3016.971679] LustreError: 97393:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 49 previous similar messages [ 3043.055129] Lustre: 97393:0:(ldlm_lib.c:2378:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3043.068256] Lustre: 97393:0:(ldlm_lib.c:2388:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3043.085753] Lustre: 97393:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client a8d1a26b-acb0-4db0-96d8-a2bc3049a32c@192.168.202.29@tcp [ 3043.108572] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3043.139412] Lustre: 97393:0:(ldlm_lib.c:1898:abort_req_replay_queue()) @@@ aborted: req@000000008394bfac x1865630405958400/t0(8589934781) o101->a8d1a26b-acb0-4db0-96d8-a2bc3049a32c@192.168.202.29@tcp:578/0 lens 592/0 e 0 to 0 dl 1779204888 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3043.185746] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3043.201630] Lustre: lustre-MDT0000: Denying connection for new client a8d1a26b-acb0-4db0-96d8-a2bc3049a32c (at 192.168.202.29@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3043.215344] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3043.265691] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3043.280576] Lustre: Skipped 2 previous similar messages [ 3043.296299] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3043.344899] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:80 to 0x280000401:577) [ 3043.345764] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:48 to 0x2c0000401:545) [ 3064.288453] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3064.292055] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3064.316927] Lustre: Skipped 1 previous similar message [ 3069.193775] Lustre: server umount lustre-OST0000 complete [ 3070.954456] LustreError: 93516:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3070.976953] LustreError: 93516:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 37 previous similar messages [ 3079.005894] Lustre: server umount lustre-OST0001 complete [ 3084.257536] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3084.269936] LustreError: Skipped 3 previous similar messages [ 3089.785536] Lustre: server umount lustre-MDT0000 complete [ 3093.402937] LustreError: 91679:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779204928 with bad export cookie 6523751853972001477 [ 3093.406118] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3093.411153] LustreError: 91679:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3093.748898] Lustre: server umount lustre-MDT0001 complete [ 3104.100079] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 11:35:37 (1779204937) [ 3113.364564] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3113.379679] Lustre: *** cfs_fail_loc=197, val=0*** [ 3128.998768] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3134.111620] LustreError: 99355:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.129@tcp: failed processing log, type 4: rc = -110 [ 3139.287510] Lustre: *** cfs_fail_loc=197, val=0*** [ 3139.289801] Lustre: Skipped 65 previous similar messages [ 3159.839269] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3159.849471] Lustre: Skipped 3 previous similar messages [ 3167.258531] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 3173.104893] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3175.521529] Lustre: server umount lustre-OST0000 complete [ 3184.935740] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 11:36:58 (1779205018) [ 3208.853340] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3209.335308] LustreError: 100865:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3209.351382] LustreError: 100865:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 3214.532195] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 3222.320921] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3226.639891] LustreError: 100896:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3226.650411] LustreError: 100896:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3226.656581] LustreError: 100896:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 17, retries 0, failed: rc = -5 [ 3226.941647] Lustre: server umount lustre-MDT0000 complete [ 3260.901853] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_hostid [ 3273.733068] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 3318.045460] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3318.258634] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3318.299188] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3318.361301] Lustre: lustre-MDT0000: new disk, initializing [ 3318.467833] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3324.349516] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 3336.736849] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3336.922411] Lustre: 104936:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3337.396169] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3342.990289] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 3352.596627] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3357.112906] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3365.998043] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3366.391975] Lustre: 106676:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3373.864966] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 3375.995624] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3383.596410] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3388.709737] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3388.947683] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3394.236918] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3394.457715] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3401.696222] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3401.703032] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3401.715456] LustreError: Skipped 1 previous similar message [ 3401.721047] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3401.721057] Lustre: Skipped 11 previous similar messages [ 3401.770819] Lustre: Skipped 6 previous similar messages [ 3402.984993] Lustre: server umount lustre-OST0000 complete [ 3413.610625] Lustre: server umount lustre-MDT0000 complete [ 3417.794832] LustreError: 104927:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779205252 with bad export cookie 6523751853972012180 [ 3417.798227] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3417.808609] LustreError: 104927:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3418.119173] Lustre: server umount lustre-MDT0001 complete [ 3428.521754] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 11:41:01 (1779205261) [ 3440.291675] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing unload_modules_local [ 3444.729281] Key type lgssc unregistered [ 3445.191522] LNet: 109004:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3445.202813] LNetError: 109004:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3445.242707] LNet: Removed LNI 192.168.202.129@tcp [ 3446.805215] Key type .llcrypt unregistered [ 3446.810105] Key type ._llcrypt unregistered [ 3469.555824] Key type ._llcrypt registered [ 3469.563363] Key type .llcrypt registered [ 3469.791913] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_hostid [ 3488.452641] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 3489.796220] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3489.931232] alg: No test for adler32 (adler32-zlib) [ 3491.192982] Lustre: Lustre: Build Version: 2.17.53_25_ge261312 [ 3491.493061] LNet: Added LNI 192.168.202.129@tcp [8/256/0/180] [ 3493.207162] Key type lgssc registered [ 3494.812067] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3540.069696] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3540.094273] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3541.335955] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3541.385584] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3541.454091] Lustre: lustre-MDT0000: new disk, initializing [ 3541.517033] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3541.529379] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3546.814867] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 3559.552169] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3559.844372] Lustre: lustre-OST0000: new disk, initializing [ 3559.847512] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3559.852213] Lustre: Skipped 1 previous similar message [ 3559.857184] Lustre: 113182:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3559.931173] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3567.292818] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 3568.772311] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3568.782525] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3568.845203] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3581.225607] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3581.419774] Lustre: lustre-OST0001: new disk, initializing [ 3581.423218] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3581.438931] Lustre: 114032:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3581.558027] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3589.450039] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 3590.789522] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3590.803607] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3590.850277] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3600.865772] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3600.884383] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3600.897130] Lustre: Skipped 1 previous similar message [ 3605.689859] Lustre: server umount lustre-MDT0000 complete [ 3613.799160] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3618.825868] LustreError: 113181:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779205454 with bad export cookie 10422089414540486314 [ 3618.826356] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3618.833884] LustreError: 113181:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3618.932967] Lustre: server umount lustre-OST0000 complete [ 3624.008592] Lustre: server umount lustre-OST0001 complete [ 3635.742887] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 11:44:29 (1779205469) [ 3650.778932] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 3680.821990] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 11:45:14 (1779205514) [ 3724.180300] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_hostid [ 3737.388438] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 3779.609034] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3780.037452] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3780.061994] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3780.120785] Lustre: lustre-MDT0000: new disk, initializing [ 3780.189547] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3780.243158] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3784.925545] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 3796.169336] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3796.400118] Lustre: 120758:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3796.491989] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3796.498163] Lustre: Skipped 1 previous similar message [ 3796.553532] Lustre: lustre-MDT0001: new disk, initializing [ 3796.640574] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3796.718051] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3796.733813] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3801.472703] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 3810.999789] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3815.403976] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3823.573871] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3823.790246] Lustre: lustre-OST0000: new disk, initializing [ 3823.792202] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3823.801877] Lustre: 122503:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3823.894535] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3830.725174] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 3833.924554] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3833.937467] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3834.019294] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3839.383884] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3845.728450] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3845.913564] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3850.071436] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3850.389316] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3854.818411] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3854.828056] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3854.830379] LustreError: Skipped 1 previous similar message [ 3854.832946] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3854.860771] Lustre: Skipped 2 previous similar messages [ 3859.240122] Lustre: server umount lustre-OST0000 complete [ 3865.056928] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3865.081028] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3865.085896] Lustre: Skipped 1 previous similar message [ 3865.110903] Lustre: Skipped 2 previous similar messages [ 3870.178023] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3870.190257] Lustre: Skipped 1 previous similar message [ 3875.299974] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3875.309649] Lustre: Skipped 1 previous similar message [ 3878.371044] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3878.697404] Lustre: server umount lustre-MDT0000 complete [ 3883.312939] LustreError: 120752:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779205718 with bad export cookie 10422089414540487539 [ 3883.315882] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3883.322206] LustreError: 120752:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3883.687851] Lustre: server umount lustre-MDT0001 complete [ 3893.173841] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 11:48:47 (1779205727) [ 3900.822291] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3901.376518] LustreError: 124767:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3901.411834] LustreError: 124767:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3901.495233] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3906.538515] LustreError: 124768:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3907.468979] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 3911.656291] LustreError: 124767:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3916.771838] LustreError: 124768:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3918.482842] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3924.128618] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 3929.922152] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3934.244357] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3941.495846] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3941.982559] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3941.984833] Lustre: Skipped 1 previous similar message [ 3948.819886] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 3955.609796] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3960.387852] Lustre: *** cfs_fail_loc=159, val=0*** [ 3960.402782] Lustre: Skipped 2 previous similar messages [ 3975.834451] Lustre: lustre-MDT0000: Client d94f00ae-e212-40ad-ac5c-6eca08209880 (at 192.168.202.29@tcp) reconnecting [ 3978.017901] Lustre: *** cfs_fail_loc=159, val=0*** [ 3978.019931] Lustre: Skipped 3 previous similar messages [ 3993.762584] Lustre: lustre-MDT0000: Client d94f00ae-e212-40ad-ac5c-6eca08209880 (at 192.168.202.29@tcp) reconnecting [ 4013.036037] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4013.043762] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4013.053193] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4013.066194] Lustre: Skipped 1 previous similar message [ 4013.096046] Lustre: Skipped 1 previous similar message [ 4017.807414] Lustre: server umount lustre-OST0000 complete [ 4023.270120] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4023.298260] Lustre: Skipped 1 previous similar message [ 4033.507096] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4033.522305] Lustre: Skipped 5 previous similar messages [ 4037.087138] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4037.314800] Lustre: server umount lustre-MDT0000 complete [ 4038.624530] LustreError: 124768:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4038.639118] LustreError: 124768:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4041.124668] LustreError: 124747:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779205876 with bad export cookie 10422089414540488421 [ 4041.133971] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4041.507937] Lustre: server umount lustre-MDT0001 complete [ 4054.463125] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing unload_modules_local [ 4057.253455] Key type lgssc unregistered [ 4057.588743] LNet: 128189:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4057.596690] LNetError: 128189:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4057.612202] LNet: Removed LNI 192.168.202.129@tcp [ 4058.549964] Key type .llcrypt unregistered [ 4058.552230] Key type ._llcrypt unregistered [ 4084.368902] Key type ._llcrypt registered [ 4084.371146] Key type .llcrypt registered [ 4084.477903] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 4085.568483] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4085.602596] alg: No test for adler32 (adler32-zlib) [ 4086.799577] Lustre: Lustre: Build Version: 2.17.53_25_ge261312 [ 4087.064705] LNet: Added LNI 192.168.202.129@tcp [8/256/0/180] [ 4088.719851] Key type lgssc registered [ 4089.994397] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4099.168330] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4099.182675] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4100.800951] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4104.950594] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 4110.319915] LustreError: 129642:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4110.355190] LustreError: 129642:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4114.953768] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4115.524652] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4121.152594] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 4127.052808] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4130.911897] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4138.116059] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4138.691984] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4145.158507] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 4145.861374] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4151.880274] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4161.182323] Lustre: *** cfs_fail_loc=159, val=0*** [ 4177.513151] Lustre: lustre-MDT0000: Client ab8c6a2b-284b-45b8-a46c-788c3baab6e6 (at 192.168.202.29@tcp) reconnecting [ 4180.582032] Lustre: *** cfs_fail_loc=159, val=0*** [ 4196.003297] Lustre: lustre-MDT0001: Client ab8c6a2b-284b-45b8-a46c-788c3baab6e6 (at 192.168.202.29@tcp) reconnecting [ 4198.288289] Lustre: *** cfs_fail_loc=159, val=0*** [ 4198.290138] Lustre: Skipped 3 previous similar messages [ 4213.861542] Lustre: lustre-MDT0001: Client ab8c6a2b-284b-45b8-a46c-788c3baab6e6 (at 192.168.202.29@tcp) reconnecting [ 4219.183745] Lustre: 132343:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4223.382025] Lustre: *** cfs_fail_loc=159, val=0*** [ 4223.388802] Lustre: Skipped 4 previous similar messages [ 4239.027210] Lustre: lustre-MDT0000: Client ff1b31da-e06d-4314-bbc9-be53a29c5bff (at 192.168.202.29@tcp) reconnecting [ 4241.796342] Lustre: *** cfs_fail_loc=159, val=0*** [ 4241.805920] Lustre: Skipped 14 previous similar messages [ 4258.441761] Lustre: lustre-MDT0000: Client ff1b31da-e06d-4314-bbc9-be53a29c5bff (at 192.168.202.29@tcp) reconnecting [ 4259.726565] Lustre: 132639:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4265.959475] LustreError: lustre-OST0000-osc-MDT0001: operation ost_destroy to node 0@lo failed: rc = -107 [ 4265.960827] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4265.975346] LustreError: Skipped 1 previous similar message [ 4266.005661] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4269.026785] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4269.036355] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4269.052561] LustreError: Skipped 4 previous similar messages [ 4269.054781] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4271.579745] Lustre: server umount lustre-OST0000 complete [ 4279.275145] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4279.283824] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4279.289316] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4279.291674] Lustre: Skipped 1 previous similar message [ 4279.325301] Lustre: Skipped 2 previous similar messages [ 4283.102151] Lustre: server umount lustre-MDT0000 complete [ 4284.385348] LustreError: 130548:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4284.409655] LustreError: 130548:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4288.020769] LustreError: 129621:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779206123 with bad export cookie 14941112491440856738 [ 4288.024040] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4288.042742] LustreError: 129621:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4288.339524] Lustre: server umount lustre-MDT0001 complete [ 4300.824912] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing unload_modules_local [ 4303.918022] Key type lgssc unregistered [ 4304.251655] LNet: 133675:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4304.257607] LNetError: 133675:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4304.278662] LNet: Removed LNI 192.168.202.129@tcp [ 4305.290723] Key type .llcrypt unregistered [ 4305.294504] Key type ._llcrypt unregistered [ 4336.340834] Key type ._llcrypt registered [ 4336.342940] Key type .llcrypt registered [ 4336.460883] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 4338.548050] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4338.578030] alg: No test for adler32 (adler32-zlib) [ 4339.840110] Lustre: Lustre: Build Version: 2.17.53_25_ge261312 [ 4340.148471] LNet: Added LNI 192.168.202.129@tcp [8/256/0/180] [ 4341.953113] Key type lgssc registered [ 4343.103945] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4351.430246] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4351.461853] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4353.182106] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4358.140750] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 4363.744364] LustreError: 135131:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4363.756791] LustreError: 135131:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4367.066278] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4367.450553] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4372.090085] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 4378.783496] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4383.246698] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4390.643771] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4391.068628] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4392.134463] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4396.552995] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 4398.277979] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4403.498443] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4411.177733] Lustre: 137373:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4414.290222] Lustre: 137422:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4416.479470] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4417.386307] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4420.739185] Lustre: 137567:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4421.566183] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4422.420292] Lustre: 137666:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4428.256377] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4428.264577] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4428.276915] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4429.287022] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4429.292117] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4431.798701] Lustre: server umount lustre-OST0000 complete [ 4439.009635] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4439.016426] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4439.034500] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4439.044572] Lustre: Skipped 1 previous similar message [ 4442.907621] Lustre: server umount lustre-MDT0000 complete [ 4444.641843] LustreError: 135127:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4444.660076] LustreError: 135127:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4447.429865] LustreError: 135114:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779206282 with bad export cookie 15796416892088774054 [ 4447.442779] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4447.453847] LustreError: 135114:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4447.965879] Lustre: server umount lustre-MDT0001 complete [ 4459.746635] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing unload_modules_local [ 4462.392747] Key type lgssc unregistered [ 4462.766281] LNet: 138711:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4462.777551] LNetError: 138711:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4463.853045] LNet: Removed LNI 192.168.202.129@tcp [ 4464.903136] Key type .llcrypt unregistered [ 4464.910463] Key type ._llcrypt unregistered [ 4491.709860] Key type ._llcrypt registered [ 4491.717441] Key type .llcrypt registered [ 4491.865789] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 4493.311584] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4493.414402] alg: No test for adler32 (adler32-zlib) [ 4494.586877] Lustre: Lustre: Build Version: 2.17.53_25_ge261312 [ 4494.799418] LNet: Added LNI 192.168.202.129@tcp [8/256/0/180] [ 4496.439259] Key type lgssc registered [ 4498.198869] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4507.622635] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4507.644402] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4509.320227] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4513.900702] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 4518.884458] LustreError: 140167:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4518.904043] LustreError: 140167:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4523.575639] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4524.090127] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4528.410917] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 4534.160760] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4539.269843] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4547.611877] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4548.172843] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4549.196817] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4554.294862] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4556.232882] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 4562.227819] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4567.749499] Lustre: *** cfs_fail_loc=159, val=0*** [ 4584.061896] Lustre: lustre-MDT0000: Client 70e463c7-b983-43dd-9bd7-9235b9a99711 (at 192.168.202.29@tcp) reconnecting [ 4605.408590] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4605.410505] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4605.414065] LustreError: Skipped 1 previous similar message [ 4605.417655] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4605.443647] Lustre: Skipped 1 previous similar message [ 4610.528815] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4610.551886] Lustre: Skipped 2 previous similar messages [ 4615.653264] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4615.671063] Lustre: Skipped 1 previous similar message [ 4616.160918] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4616.571850] Lustre: server umount lustre-OST0000 complete [ 4625.889363] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4625.897140] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4625.899805] Lustre: Skipped 1 previous similar message [ 4625.947875] Lustre: Skipped 1 previous similar message [ 4628.702234] Lustre: server umount lustre-MDT0000 complete [ 4631.016054] LustreError: 140166:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4631.046890] LustreError: 140166:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4634.097212] LustreError: 142438:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779206469 with bad export cookie 1406260463305538040 [ 4634.109768] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4634.113694] LustreError: 142438:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4634.601700] Lustre: server umount lustre-MDT0001 complete [ 4646.383227] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing unload_modules_local [ 4649.061693] Key type lgssc unregistered [ 4649.340576] LNet: 143487:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4649.346118] LNetError: 143487:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4650.405279] LNet: Removed LNI 192.168.202.129@tcp [ 4651.387725] Key type .llcrypt unregistered [ 4651.390636] Key type ._llcrypt unregistered [ 4677.801714] Key type ._llcrypt registered [ 4677.805616] Key type .llcrypt registered [ 4677.898536] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 4678.915130] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4678.964985] alg: No test for adler32 (adler32-zlib) [ 4680.229921] Lustre: Lustre: Build Version: 2.17.53_25_ge261312 [ 4680.480728] LNet: Added LNI 192.168.202.129@tcp [8/256/0/180] [ 4682.199210] Key type lgssc registered [ 4683.681323] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4692.685573] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4692.723635] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4694.376688] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4699.421464] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 4704.737142] LustreError: 144944:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4704.752658] LustreError: 144944:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4709.116438] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4709.696649] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4714.298738] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 4720.782953] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4725.371680] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4732.715362] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4733.107742] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4734.140780] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4739.278917] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4740.312338] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 4747.113649] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4769.107117] Lustre: lustre-MDT0001: evicting c210f4fb-62ef-4e6b-8139-3f98b609c50c (at 192.168.202.29@tcp) by administrative request [ 4769.112741] Lustre: Skipped 1 previous similar message [ 4800.992992] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4800.993149] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4801.011469] LustreError: Skipped 1 previous similar message [ 4801.012264] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4801.062211] Lustre: Skipped 1 previous similar message [ 4803.689990] Lustre: server umount lustre-OST0000 complete [ 4809.186397] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4809.209948] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4809.212811] Lustre: Skipped 2 previous similar messages [ 4814.315902] LustreError: 144943:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4814.490455] Lustre: server umount lustre-MDT0000 complete [ 4818.110244] LustreError: 144925:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779206653 with bad export cookie 8958853541838102141 [ 4818.115672] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4818.126770] LustreError: 144925:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4818.928813] Lustre: server umount lustre-MDT0001 complete [ 4830.529322] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing unload_modules_local [ 4833.734258] Key type lgssc unregistered [ 4833.981956] LNet: 148658:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4833.986842] LNetError: 148658:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4835.056610] LNet: Removed LNI 192.168.202.129@tcp [ 4836.009335] Key type .llcrypt unregistered [ 4836.011364] Key type ._llcrypt unregistered [ 4919.830692] Key type ._llcrypt registered [ 4919.835358] Key type .llcrypt registered [ 4920.078957] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 4921.504662] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4921.805084] alg: No test for adler32 (adler32-zlib) [ 4923.190492] Lustre: Lustre: Build Version: 2.17.53_25_ge261312 [ 4923.475973] LNet: Added LNI 192.168.202.129@tcp [8/256/0/180] [ 4925.271167] Key type lgssc registered [ 4926.999844] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4936.725896] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4936.759788] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4938.542782] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4943.712431] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 4947.954731] LustreError: 151189:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4947.973723] LustreError: 151189:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4953.060387] LustreError: 151190:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4954.204375] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4954.861194] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4960.326421] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 4966.719563] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4971.520924] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4978.396895] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4978.818849] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4979.854359] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4982.904900] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4985.622179] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 4994.085711] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5005.794310] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5005.803389] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5005.829661] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5008.871058] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5008.873440] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5008.880670] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5010.092647] Lustre: server umount lustre-OST0000 complete [ 5016.034230] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5016.043901] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5016.087073] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5016.096579] Lustre: Skipped 1 previous similar message [ 5019.105354] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5019.133502] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5020.933754] Lustre: server umount lustre-MDT0000 complete [ 5024.225257] LustreError: 151190:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5024.236776] LustreError: 151190:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5024.857233] LustreError: 151169:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779206860 with bad export cookie 14434115614635683723 [ 5024.874701] LustreError: 151169:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5024.881256] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5025.433950] Lustre: server umount lustre-MDT0001 complete [ 5039.509746] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing unload_modules_local [ 5042.468521] Key type lgssc unregistered [ 5042.813082] LNet: 154375:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5042.824574] LNetError: 154375:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5042.843994] LNet: Removed LNI 192.168.202.129@tcp [ 5043.919923] Key type .llcrypt unregistered [ 5043.922257] Key type ._llcrypt unregistered [ 5072.038425] Key type ._llcrypt registered [ 5072.040193] Key type .llcrypt registered [ 5072.205690] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 5073.489173] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5073.765080] alg: No test for adler32 (adler32-zlib) [ 5074.885912] Lustre: Lustre: Build Version: 2.17.53_25_ge261312 [ 5075.155265] LNet: Added LNI 192.168.202.129@tcp [8/256/0/180] [ 5076.895183] Key type lgssc registered [ 5078.537683] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5098.658668] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 12:08:52 (1779206932) [ 5118.521453] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_hostid [ 5130.729667] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 5171.811375] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5171.836759] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5173.130518] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5173.160173] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5173.259658] Lustre: lustre-MDT0000: new disk, initializing [ 5173.344135] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5173.369260] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5178.043939] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 5190.879578] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5191.063937] Lustre: 158665:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5191.160973] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5191.173673] Lustre: Skipped 1 previous similar message [ 5191.338715] Lustre: lustre-MDT0001: new disk, initializing [ 5191.454551] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5191.509182] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5191.522098] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5196.569596] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 5205.005949] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5209.358753] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5216.527563] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5216.866649] Lustre: lustre-OST0000: new disk, initializing [ 5216.873932] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5216.882553] Lustre: 160409:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5217.059240] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5223.527558] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5223.554081] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5223.697460] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5224.011268] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 5232.895862] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5249.514096] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5249.527583] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5249.532821] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5249.537539] Lustre: Skipped 1 previous similar message [ 5249.545078] Lustre: Skipped 1 previous similar message [ 5252.247231] Lustre: server umount lustre-OST0000 complete [ 5258.209947] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5258.216908] LustreError: Skipped 1 previous similar message [ 5258.228076] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5258.254966] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5259.746579] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5259.747738] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5263.149346] Lustre: server umount lustre-MDT0000 complete [ 5267.421595] LustreError: 159444:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779207102 with bad export cookie 11100177284314606100 [ 5267.426040] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5267.431083] LustreError: 159444:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5267.851453] Lustre: server umount lustre-MDT0001 complete [ 5279.133695] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing unload_modules_local [ 5281.826256] Key type lgssc unregistered [ 5281.986493] LNet: 162184:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5281.991602] LNetError: 162184:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5282.005149] LNet: Removed LNI 192.168.202.129@tcp [ 5282.976965] Key type .llcrypt unregistered [ 5282.978686] Key type ._llcrypt unregistered [ 5311.716583] Key type ._llcrypt registered [ 5311.718888] Key type .llcrypt registered [ 5311.833525] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 5313.048871] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5313.062213] alg: No test for adler32 (adler32-zlib) [ 5314.268279] Lustre: Lustre: Build Version: 2.17.53_25_ge261312 [ 5314.545336] LNet: Added LNI 192.168.202.129@tcp [8/256/0/180] [ 5316.271801] Key type lgssc registered [ 5317.730844] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5327.166716] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5327.200920] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5328.919951] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5333.313531] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 5339.117216] LustreError: 163637:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5339.130473] LustreError: 163637:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5343.914270] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5344.226872] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 5344.322164] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5350.735352] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 5358.665303] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5362.790774] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5370.613545] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5371.118537] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5374.198312] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5379.147261] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 5386.657637] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5397.270079] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5397.292861] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5397.319588] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5397.330554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.354740] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5398.531133] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5398.568540] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5398.602523] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5398.619237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.640072] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5399.707496] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5399.730204] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5399.742427] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5399.751419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.769091] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5400.815932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.834916] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5401.844320] Lustre: Skipped 1 previous similar message [ 5401.863371] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5401.879407] Lustre: Skipped 1 previous similar message [ 5401.885585] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5401.895126] Lustre: Skipped 1 previous similar message [ 5401.899259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.905523] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5401.916580] Lustre: Skipped 1 previous similar message [ 5402.861571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.028311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.875428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.940196] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5405.947906] Lustre: Skipped 3 previous similar messages [ 5405.969580] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5405.983536] Lustre: Skipped 3 previous similar messages [ 5405.990562] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5405.996761] Lustre: Skipped 3 previous similar messages [ 5406.001713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.023253] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5406.034984] Lustre: Skipped 3 previous similar messages [ 5407.113418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.446745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.604164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.647427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.904986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.071325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.141250] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5414.152264] Lustre: Skipped 6 previous similar messages [ 5414.174332] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5414.187748] Lustre: Skipped 6 previous similar messages [ 5414.191696] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5414.196411] Lustre: Skipped 6 previous similar messages [ 5414.201356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.210470] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5414.213963] Lustre: Skipped 6 previous similar messages [ 5415.136779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.036547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.094743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.840249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.765825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.724733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.637938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.497871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.468790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.442202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.158878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.065265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.979366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.051547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.910490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.714745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.419982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.181050] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5430.185653] Lustre: Skipped 17 previous similar messages [ 5430.201297] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5430.208807] Lustre: Skipped 17 previous similar messages [ 5430.212587] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5430.220769] Lustre: Skipped 17 previous similar messages [ 5430.228379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.238286] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5430.246176] Lustre: Skipped 17 previous similar messages [ 5430.955114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.748193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.567190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.633617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.578321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.557305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.602722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.505891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.365059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.237571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.283361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.160739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.936039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.693547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.407609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.304430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.266986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.986752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.926341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.977964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.840516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.793328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.675204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.469333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.570541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.410785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.294922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.086660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.959159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.636202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.652694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.447545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.581818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.907539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.357936] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5462.369377] Lustre: Skipped 34 previous similar messages [ 5462.395133] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5462.403728] Lustre: Skipped 34 previous similar messages [ 5462.416389] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5462.419555] Lustre: Skipped 34 previous similar messages [ 5462.428286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.445856] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5462.452504] Lustre: Skipped 34 previous similar messages [ 5463.684411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.794647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.899135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.906401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.223457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.355872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.193402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.399757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.387276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.571076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.561320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.669262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.964726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.290757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.427570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.650838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.919565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.070454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.505942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.452805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.286893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.319505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.256163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.238154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.085852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.023507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.905931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.906116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.652895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.690417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.071309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.096813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.880842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.722148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.621572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.385115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.181892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.974040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.870456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.680696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.677351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.584190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.555464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.441324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.440462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.565220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.651182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.840685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.692718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.919578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.051642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.906576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.262627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.644863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.996679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.421800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.471232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.554579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.485730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.039972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.645723] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5527.654432] Lustre: Skipped 60 previous similar messages [ 5527.683431] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5527.695837] Lustre: Skipped 60 previous similar messages [ 5527.700573] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5527.703499] Lustre: Skipped 60 previous similar messages [ 5527.714623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.738295] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5527.745164] Lustre: Skipped 60 previous similar messages [ 5529.141431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.533917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.185422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.361208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.889490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.867187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.236035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.733053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.343593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.682623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.388937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.540640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.844421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.987456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.494712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.786808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.985638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.982405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.110835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.165553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.183407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.039984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.991635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.092636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.190075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.162590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.170819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.955624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.010714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.863777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.888072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.475534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.821051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.952795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.879046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.154975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.222951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.278293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.341047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.283585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.385665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.215662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.184348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.967462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.910511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.712227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.650873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.546467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.436473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.371933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.301224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.365467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.547289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.336764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.155491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.984045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.861817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.593328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.584646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.509555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.521708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.334462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.329466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.224561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.421964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.602433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.721539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.725822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.712694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.652218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.691619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.534825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.661076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.664491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.885769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.087059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.981723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.198939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.067313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.234834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.174728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.699514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.839399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.133981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.385595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.287410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.572657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.326843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.830104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.441116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.279146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.282592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.092327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.125590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.186638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.744695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.229955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.339233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.004288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.733707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.938430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.913156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.932933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.847924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.916800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.117400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.160469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.599381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.868533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.332559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.556638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.548365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.520866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.567927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.855827] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5656.879359] Lustre: Skipped 114 previous similar messages [ 5656.908749] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5656.930359] Lustre: Skipped 114 previous similar messages [ 5656.934262] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5656.941533] Lustre: Skipped 114 previous similar messages [ 5656.948768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.962678] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5656.974118] Lustre: Skipped 114 previous similar messages [ 5658.223090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.149870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.941423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.928557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.957336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.177144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.430980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.949158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.802863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.048771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.250359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.448676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.635672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.454060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.432755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.576395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.609185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.736377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.622341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.674706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.769636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.030364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.062901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.106407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.138250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.199953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.281279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.685960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.885184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.862230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.823486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.870924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.609424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.711991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.747083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.791278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.836263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.976867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.131125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.102354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.012208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.079533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.083711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.192190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.107294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.268443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.483940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.442851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.427451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.142019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.005404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.885275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.199274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.186291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.351756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.440679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.751106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.905829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.361526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.922638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.143373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.548401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.167101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.468416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.595575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.898235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.318764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.701276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.899693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.075129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.200326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.438718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.462831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.746351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.057987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.130947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.109857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.609967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.180847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.561602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.886977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.223503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.183748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.233869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.180505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.562138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.057484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.517980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.708765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.058077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.020748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.193659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.375417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.094673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.177350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.094785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.352965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.273660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.411425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.371493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.141869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.913830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.216086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.403871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.410598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.417248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.252519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.148478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.139250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.904125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.651488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.716845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.681991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.783309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.841844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.681620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.602932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.592126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.589483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.569988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.508763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.410597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.367069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.280890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.278541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.364417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.305710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.212963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.226045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.203682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.298673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.093151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.137439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.031358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.069509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.128592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.178698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.080463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.108498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.097269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.137252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.974366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.932698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.813268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.805749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.804624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.756252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.571170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.054711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.236215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.439228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.571426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.548245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.595194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.448973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.547121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.638456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.768792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.692686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.616164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.601480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.614257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.743643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.983293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.399749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.879207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.849446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.919119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.093993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.049981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.954029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.706901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.557212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.542170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.605658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.434329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.471409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.584718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.632963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.837851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.784332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.776039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.884885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.838575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.076583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.111822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.982507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.037698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.955103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.063904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.177203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.447769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.833477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.699787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.679838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.681921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.641959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.484618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.685959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.894819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.941601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.845292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.897980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.826367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.524926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.499263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.205530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.288640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.440324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.650559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.750177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.647909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.723832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.585288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.595311] LustreError: 163657:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5886.617214] LustreError: 163657:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5886.640546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.721426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.789363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.899782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.212640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.460793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.590929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.663282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.934106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.309607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.488387] LustreError: 163657:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5898.503160] LustreError: 163657:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5898.517640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.840189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.018540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.222973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.123546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.276762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.438654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.775573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.991250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.137513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.305494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.642883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.536899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.535792] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5913.550048] Lustre: Skipped 237 previous similar messages [ 5913.567758] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5913.577893] Lustre: Skipped 237 previous similar messages [ 5913.585084] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5913.588106] Lustre: Skipped 237 previous similar messages [ 5913.592886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.604926] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5913.610903] Lustre: Skipped 237 previous similar messages [ 5914.533495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.598884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.504662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.749071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.722593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.870732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.707558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.826628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.157090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.171459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.152831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.134482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.523752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.801609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.022963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.343589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.647124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.051034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.325223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.696968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.718752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.846339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.042872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.291909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.425328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.416212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.606553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.445330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.542687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.389980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.648871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.948340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.391370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.476937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.729363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.450613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.391144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.474488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.451142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.511463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.470202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.796454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.813946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.971445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.249361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.380582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.235588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.400850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.391843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.469970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.359783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.352831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.335052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.222192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.992174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.752439] LustreError: 163657:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5974.764165] LustreError: 163657:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5974.765831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.696920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.463534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.421848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.514342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.409891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.299116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.285882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.268310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.161471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.139887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.266514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.141277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.019174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.884822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.823112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.682890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.612187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.506239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.457459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.305076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.304347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.549312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.646990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.508721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.538641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.431488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.302073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.143485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.033348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.837744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.703617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.487228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.510759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.362068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.344296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.171720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.149797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.223890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.351953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.567466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.307666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.367384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.395967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.461037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.703734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.932430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.099712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.652729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.906990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.046821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.507416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.779414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.755655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.914640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.951341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.191381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.139726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.129304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.542979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.790622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.721728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.876964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.740919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.714235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.592390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.564470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.772300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.950803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.855229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.068963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.379992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.462535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.025912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.312090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.882194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.212419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.480988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.746808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.950550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.935471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.128178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.941651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.024300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.067927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.046928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.888166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.013071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.943827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.948765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.729708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.789216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.798746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.746219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.687343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.968677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.158378] LustreError: 163657:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6075.168506] LustreError: 163657:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6075.178229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.338969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.320060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.134579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.243983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.124225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.196549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.300272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.319828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.310854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.231685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.170762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.178879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.300543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.144176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.984020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.746717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.579322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.827609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.722474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.555853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.238910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.259085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.386734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.626879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.579058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.939294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.020871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.883861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.683270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.560835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.757313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.804688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.888964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.072612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.070460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.117675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.236955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.550361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.809496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.677558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.707646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.488491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.546829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.440697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.362666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.145732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.338314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.316917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.652700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.792353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.337097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.360960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.627789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.696248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.791544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.802508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.774780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.669271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.693052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.710638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.677376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.647452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.385050] LustreError: 163657:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6139.405237] LustreError: 163657:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6139.428154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.583570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.414446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.346251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.148034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.871406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.069696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.073941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.137066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.094958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.056871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.833915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.565328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.636540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.955947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.466890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.473274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.502162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.530695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.805080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.790523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.732697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.877949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.767317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.720704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.743087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.113734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.074349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.851336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.749791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.823297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.723300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.777879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.835752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.802185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.705540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.587359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.496309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.437274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.588317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.807944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.204258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.223867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.379030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.298435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.543853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.437979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.680363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.724887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.025663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.155631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.258087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.373189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.341277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.450276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.548888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.617725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.441353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.313425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.392698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.495822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.410494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.400630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.779081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.130390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.463559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.806811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.102774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.212037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.110169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.952756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.920498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.069431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.058165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.959699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.874051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.949453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.073679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.070415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.210627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.055786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.044774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.927880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.001312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.072532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.082846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.174661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.212818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.181057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.181312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.451641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.620540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.649734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.493969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.407330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.515604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.459567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.517974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.514852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.664833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.731371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.802762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.747761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.697677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.491925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.519470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.518513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.446109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.400243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.327696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.259895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.382984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.323669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.366373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.372383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.398648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.478237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.381522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.382157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.298756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.344596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.236039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.329767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.305705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.256053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.082976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.940264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.851704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.752173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.761067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.042968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.337656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.347152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.390313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.172381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.920849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.690877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.558944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.384137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.567679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.843626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.745443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.701779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.735793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.990850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.951540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.982881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.896524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.788405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.793696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.791145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.786905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.836736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.673674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.766603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.898600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.982205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.903611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.818509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.656757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.800679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.959726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.055943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.953565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.872729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.815261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.822948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.787806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.850655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.665431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.745374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.771608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.811530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.893342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.862930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.804705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.457500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6319.481360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.626853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.634323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.684728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.723461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.866687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.918211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.671496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.496185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.437467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.581923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.768236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6331.809368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.949395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.074402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.317768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.377766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.347644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6338.589905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.769474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.933406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.940443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6343.004625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6343.810363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.587890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.689709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.716601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.791195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6348.733582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.746575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.632777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6351.793143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.882304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.711415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6354.783252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.785486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.697776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.737110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.828724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.862098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.960474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6362.141425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.386460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.640911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.884967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.056706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.078621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.225075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.234823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.284813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.143059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6373.214547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.237362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.142932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.937475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.932705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6377.770195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.956288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.857783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.746866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.647414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6382.698247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.778367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6385.046721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6385.943172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.112733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6388.280402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6389.505909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6390.891304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.864856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.035182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.943162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6394.862488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6395.761529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6396.734112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.381528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.332336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6399.170502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.225867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6401.062841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6401.995982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.886645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6403.807898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6404.745534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.597434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6406.464551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6407.463778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6408.351710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6409.487450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.300326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.414824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6412.408635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.421197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6414.398983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.436555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.221312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.189612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6418.037827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.029288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.146881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6421.084697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6422.115966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6422.931457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6423.986986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.850985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.834124] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6425.839271] Lustre: Skipped 496 previous similar messages [ 6425.855462] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6425.864805] Lustre: Skipped 496 previous similar messages [ 6425.869276] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6425.872034] Lustre: Skipped 496 previous similar messages [ 6425.876723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.886785] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6425.897358] Lustre: Skipped 496 previous similar messages [ 6426.847683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6427.697356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6428.796971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6429.733609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.843503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.937521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6432.977319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6433.937564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6434.775693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.712040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6436.829422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6437.877585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.819180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6439.782704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6440.582177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.475067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.388258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6443.452600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6444.237261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.207856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6792.462222] LustreError: 165236:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6792.468068] LustreError: 163657:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7228.897122] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7228.903148] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7228.929817] Lustre: Skipped 20 previous similar messages [ 7228.935019] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7230.950591] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7230.961230] Lustre: Skipped 1 previous similar message [ 7233.989900] Lustre: server umount lustre-OST0000 complete [ 7238.632636] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7238.655522] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7241.186715] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7241.192109] Lustre: Skipped 1 previous similar message [ 7245.066451] Lustre: server umount lustre-MDT0000 complete [ 7246.305613] LustreError: 163637:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7246.322205] LustreError: 163637:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7249.060565] LustreError: 163618:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779209084 with bad export cookie 18399832691558316399 [ 7249.071328] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7249.073075] LustreError: 163618:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7249.468430] Lustre: server umount lustre-MDT0001 complete [ 7262.330090] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing unload_modules_local [ 7265.966540] Key type lgssc unregistered [ 7266.315634] LNet: 218511:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7266.324705] LNetError: 218511:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7267.365519] LNet: Removed LNI 192.168.202.129@tcp [ 7268.403127] Key type .llcrypt unregistered [ 7268.408428] Key type ._llcrypt unregistered [ 7293.694721] Key type ._llcrypt registered [ 7293.700443] Key type .llcrypt registered [ 7293.828119] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 7295.195605] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7295.387036] alg: No test for adler32 (adler32-zlib) [ 7296.458607] Lustre: Lustre: Build Version: 2.17.53_25_ge261312 [ 7296.728561] LNet: Added LNI 192.168.202.129@tcp [8/256/0/180] [ 7298.407242] Key type lgssc registered [ 7299.588384] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7306.723986] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7306.749314] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7308.331568] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7312.686902] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 7318.503807] LustreError: 219983:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7318.527218] LustreError: 219983:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7322.476607] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7322.796567] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7327.537563] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 7334.191546] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7337.911576] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7345.657476] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7346.142486] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7352.754234] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 7353.378374] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7360.169704] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7365.181011] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7367.200324] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7367.222494] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7367.241454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7367.255464] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7367.257390] LustreError: 220003:0:(osp_precreate.c:1264:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7372.385652] Lustre: lustre-OST0000: Not available for connect from 192.168.202.29@tcp (stopping) [ 7372.769110] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7372.777980] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7373.796268] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7373.801029] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7373.807809] Lustre: Skipped 1 previous similar message [ 7375.888427] Lustre: server umount lustre-OST0000 complete [ 7387.287128] LustreError: 220003:0:(osp_precreate.c:1264:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7412.612272] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7413.106761] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7414.120996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7414.142844] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7421.559755] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 7424.455189] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7424.473612] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 7427.958306] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7439.942074] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 9 sec [ 7443.772984] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffffa0ba05e8d000.ost_server_uuid 50 [ 7445.364129] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffffa0ba05e8d000.ost_server_uuid in FULL state after 0 sec [ 7455.209503] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7455.213071] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7455.249833] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7455.252688] Lustre: Skipped 1 previous similar message [ 7457.190920] Lustre: server umount lustre-OST0000 complete [ 7465.441632] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7465.444018] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7465.472394] Lustre: Skipped 2 previous similar messages [ 7469.072475] Lustre: server umount lustre-MDT0000 complete [ 7470.573060] LustreError: 220694:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7470.588834] LustreError: 220694:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7473.071193] LustreError: 219963:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779209308 with bad export cookie 6543624680585983394 [ 7473.085712] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7473.488614] Lustre: server umount lustre-MDT0001 complete [ 7486.709824] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing unload_modules_local [ 7489.946789] Key type lgssc unregistered [ 7490.261318] LNet: 224648:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7490.267057] LNetError: 224648:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7490.297889] LNet: Removed LNI 192.168.202.129@tcp [ 7491.253894] Key type .llcrypt unregistered [ 7491.256130] Key type ._llcrypt unregistered [ 7523.901412] Key type ._llcrypt registered [ 7523.907807] Key type .llcrypt registered [ 7524.085913] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 7525.519913] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7525.705104] alg: No test for adler32 (adler32-zlib) [ 7526.780892] Lustre: Lustre: Build Version: 2.17.53_25_ge261312 [ 7527.016552] LNet: Added LNI 192.168.202.129@tcp [8/256/0/180] [ 7528.695179] Key type lgssc registered [ 7529.834718] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7540.008249] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7540.019574] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7541.452839] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7546.204239] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 7550.946710] LustreError: 226416:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7550.959283] LustreError: 226416:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7554.196334] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7554.600256] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7559.047185] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 7562.505847] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7569.978874] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7570.311923] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7576.767955] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 7586.604990] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7586.744849] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7586.801772] Lustre: lustre-OST0001: new disk, initializing [ 7586.809382] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7586.823131] Lustre: 228672:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7586.949459] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7593.192912] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 7593.655647] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7593.669882] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7593.764086] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7603.610401] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7613.890674] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7629.796220] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7629.805596] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7629.817828] Lustre: Skipped 2 previous similar messages [ 7629.825512] Lustre: Skipped 3 previous similar messages [ 7634.916676] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7636.014762] Lustre: server umount lustre-MDT0000 complete [ 7639.721976] LustreError: 229540:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779209474 with bad export cookie 18316327905095249819 [ 7639.730944] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7639.744900] LustreError: 229540:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7640.039551] LustreError: 227588:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7640.042855] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7640.069266] LustreError: 227588:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7640.110336] Lustre: Skipped 1 previous similar message [ 7640.355464] Lustre: server umount lustre-MDT0001 complete [ 7647.459776] Lustre: server umount lustre-OST0000 complete [ 7652.910230] Lustre: server umount lustre-OST0001 complete [ 7678.796528] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 7690.864726] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7691.481761] LustreError: 231755:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7691.514273] LustreError: 231755:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7691.577951] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7696.345340] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 7696.864659] LustreError: 231756:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7700.969961] LustreError: 231755:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7705.979665] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7706.581283] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7711.311970] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 7714.596192] Lustre: 232864:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7722.682189] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7723.225401] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7731.391418] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 7732.471305] LustreError: 233218:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7732.508973] LustreError: 233218:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7737.585286] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49760) [ 7741.812417] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7747.594725] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 7750.786556] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 7762.578206] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7767.271233] Lustre: 234700:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7783.414140] Lustre: mylustre-MDT0000-lwp-OST0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7783.426679] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7783.455564] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7783.480416] Lustre: Skipped 4 previous similar messages [ 7783.526242] Lustre: Skipped 6 previous similar messages [ 7786.748849] Lustre: server umount mylustre-MDT0000 complete [ 7788.524602] LustreError: 234230:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7788.542408] LustreError: 234230:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 7790.744176] LustreError: 231732:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779209625 with bad export cookie 18316327905095252626 [ 7790.752827] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7790.770734] LustreError: 231732:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7791.148740] Lustre: server umount mylustre-MDT0001 complete [ 7805.927280] Lustre: server umount mylustre-OST0000 complete [ 7808.993645] Lustre: 225646:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779209628/real 1779209628] req@ffff8a8870a2a680 x1865636236125184/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779209644 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7809.031848] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7810.418744] Lustre: server umount mylustre-OST0001 complete [ 7836.744291] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 7849.189498] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7849.752372] LustreError: 237000:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7849.826614] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7849.836852] Lustre: Skipped 1 previous similar message [ 7855.438378] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 7865.862310] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7870.718805] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 7874.155651] Lustre: 238109:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7880.854776] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7887.387850] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 7894.669825] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49792) [ 7896.472418] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7902.215052] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 7903.973645] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 7912.004703] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7915.737149] Lustre: 239944:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7922.658409] Lustre: tfs-MDT0000-lwp-OST0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7922.664546] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7922.666200] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7922.674064] Lustre: Skipped 1 previous similar message [ 7924.879841] Lustre: server umount tfs-MDT0000 complete [ 7927.778508] LustreError: 237001:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7927.812255] LustreError: 237001:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 9 previous similar messages [ 7929.173651] LustreError: 236975:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779209764 with bad export cookie 18316327905095254516 [ 7929.178430] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7929.183431] LustreError: 236975:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7929.549337] Lustre: server umount tfs-MDT0001 complete [ 7943.170318] Lustre: server umount tfs-OST0000 complete [ 7957.444880] Lustre: server umount tfs-OST0001 complete [ 7980.073950] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 7991.632055] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7992.361372] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7992.376439] Lustre: Skipped 3 previous similar messages [ 7996.991671] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 8006.008864] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8011.003259] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 8013.905895] Lustre: 243292:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8020.054809] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8026.823961] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 8035.805527] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8037.125891] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 8037.141304] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49824) [ 8042.503293] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 8050.744654] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8054.251511] Lustre: 245132:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8061.842979] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 12:58:15 (1779209895) [ 8078.304287] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8078.310233] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8078.322586] Lustre: Skipped 1 previous similar message [ 8078.327885] Lustre: Skipped 3 previous similar messages [ 8078.337047] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8084.563922] Lustre: server umount lustre-MDT0000 complete [ 8088.377479] LustreError: 242163:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779209923 with bad export cookie 18316327905095256182 [ 8088.381506] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8088.388474] LustreError: 242163:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8088.544129] LustreError: 242181:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8088.548442] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8088.562430] LustreError: 242181:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 8088.580278] Lustre: Skipped 8 previous similar messages [ 8095.030547] Lustre: server umount lustre-MDT0001 complete [ 8106.686781] Lustre: server umount lustre-OST0000 complete [ 8118.059502] Lustre: server umount lustre-OST0001 complete [ 8126.628196] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_hostid [ 8137.147263] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 8178.783524] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 8178.783524] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8178.783524] [ 8178.798282] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8179.046334] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8179.096533] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8179.203302] Lustre: lustre-MDT0000: new disk, initializing [ 8179.273889] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8179.285166] Lustre: Skipped 3 previous similar messages [ 8179.299276] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8183.621917] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 8193.995717] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 8193.995717] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8193.995717] [ 8194.045349] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8194.253175] Lustre: 248291:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8194.388135] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8194.396132] Lustre: Skipped 1 previous similar message [ 8194.538851] Lustre: lustre-MDT0001: new disk, initializing [ 8194.706125] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8194.717862] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8199.343139] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 8210.407783] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8210.779772] Lustre: lustre-OST0000: new disk, initializing [ 8210.783218] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8210.790706] Lustre: 249839:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8212.382793] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8212.398967] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8212.526940] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8216.912142] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 8227.557266] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8227.792375] Lustre: lustre-OST0001: new disk, initializing [ 8227.801371] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8227.814973] Lustre: 250707:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8228.981288] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8228.995612] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8229.085147] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8234.350744] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 8247.009710] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 13:01:20 (1779210080) [ 8262.050811] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 13:01:35 (1779210095) [ 8279.008561] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8279.013466] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8279.019599] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8279.024121] Lustre: Skipped 5 previous similar messages [ 8279.042167] Lustre: Skipped 3 previous similar messages [ 8283.714393] Lustre: server umount lustre-OST0000 complete [ 8295.160261] Lustre: server umount lustre-MDT0000 complete [ 8299.860736] LustreError: 250706:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779210135 with bad export cookie 18316327905095257736 [ 8299.866667] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8299.872907] LustreError: 250706:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 8314.335190] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8314.861907] Lustre: server umount lustre-MDT0001 complete [ 8329.872238] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing unload_modules_local [ 8343.691874] Lustre: server umount lustre-OST0001 complete [ 8349.923596] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_hostid [ 8367.207575] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 8406.254367] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8406.566460] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8406.603671] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8406.697533] Lustre: lustre-MDT0000: new disk, initializing [ 8406.804472] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8410.947074] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 8422.280473] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8422.478456] Lustre: 255208:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8422.771159] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8427.236698] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 8434.844843] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8438.629715] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8445.939309] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8446.313628] Lustre: lustre-OST0000: new disk, initializing [ 8446.322282] Lustre: Skipped 1 previous similar message [ 8446.348327] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8446.359266] Lustre: Skipped 2 previous similar messages [ 8446.364794] Lustre: 256947:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8446.428545] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8446.434495] Lustre: Skipped 5 previous similar messages [ 8447.788289] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8447.794835] Lustre: Skipped 1 previous similar message [ 8447.802281] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8447.963519] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8453.598991] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 8461.876624] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8468.450928] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8468.451156] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8468.454685] LustreError: Skipped 2 previous similar messages [ 8468.457031] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8468.461713] Lustre: Skipped 5 previous similar messages [ 8468.469368] Lustre: Skipped 9 previous similar messages [ 8474.210846] Lustre: server umount lustre-OST0000 complete [ 8484.589680] Lustre: server umount lustre-MDT0000 complete [ 8488.242615] LustreError: 255200:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779210323 with bad export cookie 18316327905095259206 [ 8488.245057] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8488.258201] LustreError: 255200:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8488.873377] Lustre: server umount lustre-MDT0001 complete [ 8502.203381] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing unload_modules_local [ 8505.262826] Key type lgssc unregistered [ 8505.570928] LNet: 258576:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8505.579748] LNetError: 258576:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8506.605804] LNet: Removed LNI 192.168.202.129@tcp [ 8507.535164] Key type .llcrypt unregistered [ 8507.536826] Key type ._llcrypt unregistered [ 8539.240632] Key type ._llcrypt registered [ 8539.244934] Key type .llcrypt registered [ 8539.402873] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 8540.563047] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8540.633208] alg: No test for adler32 (adler32-zlib) [ 8542.154270] Lustre: Lustre: Build Version: 2.17.53_25_ge261312 [ 8542.512725] LNet: Added LNI 192.168.202.129@tcp [8/256/0/180] [ 8544.311256] Key type lgssc registered [ 8545.873944] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8555.303100] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8555.353790] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8557.069167] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8562.053776] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 8567.280827] LustreError: 260078:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8567.320865] LustreError: 260078:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8571.775610] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8572.226826] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8577.365577] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 8582.597646] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8586.705648] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8593.974587] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8594.559795] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8601.304559] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 8602.862017] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8608.063779] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8609.366521] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8613.344241] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8613.361276] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8613.374456] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8617.768799] Lustre: server umount lustre-OST0000 complete [ 8623.586196] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8623.593632] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8623.601717] LustreError: Skipped 1 previous similar message [ 8623.606272] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8623.624924] Lustre: Skipped 2 previous similar messages [ 8623.640924] Lustre: Skipped 2 previous similar messages [ 8628.728628] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8628.733969] Lustre: Skipped 1 previous similar message [ 8628.950935] Lustre: server umount lustre-MDT0000 complete [ 8633.227732] LustreError: 260059:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779210468 with bad export cookie 14558984399389414219 [ 8633.245603] LustreError: 260059:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8633.246197] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8633.699687] Lustre: server umount lustre-MDT0001 complete [ 8645.492343] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing unload_modules_local [ 8648.633030] Key type lgssc unregistered [ 8649.013489] LNet: 263307:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8649.022814] LNetError: 263307:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8649.042392] LNet: Removed LNI 192.168.202.129@tcp [ 8649.932269] Key type .llcrypt unregistered [ 8649.934059] Key type ._llcrypt unregistered [ 8669.820357] Key type ._llcrypt registered [ 8669.828095] Key type .llcrypt registered [ 8669.998697] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 8671.749145] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8671.834695] alg: No test for adler32 (adler32-zlib) [ 8672.944395] Lustre: Lustre: Build Version: 2.17.53_25_ge261312 [ 8673.237468] LNet: Added LNI 192.168.202.129@tcp [8/256/0/180] [ 8674.999213] Key type lgssc registered [ 8676.325587] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8686.626081] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8686.636903] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8702.111371] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8707.231236] LustreError: 264540:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.129@tcp: failed processing log, type 4: rc = -110 [ 8734.047194] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8740.963954] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 8747.731334] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8750.134871] Lustre: server umount lustre-OST0000 complete [ 8766.766703] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing unload_modules_local [ 8769.186463] Key type lgssc unregistered [ 8769.511989] LNet: 266041:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8769.524654] LNetError: 266041:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8769.551142] LNet: Removed LNI 192.168.202.129@tcp [ 8770.515143] Key type .llcrypt unregistered [ 8770.527703] Key type ._llcrypt unregistered [ 8809.040364] Key type ._llcrypt registered [ 8809.043390] Key type .llcrypt registered [ 8809.164497] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 8810.192752] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8810.225099] alg: No test for adler32 (adler32-zlib) [ 8811.392349] Lustre: Lustre: Build Version: 2.17.53_25_ge261312 [ 8811.666730] LNet: Added LNI 192.168.202.129@tcp [8/256/0/180] [ 8813.323239] Key type lgssc registered [ 8814.346499] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8827.303873] loop0: detected capacity change from 0 to 204800000 [ 8827.508388] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8827.523501] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8827.735143] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8828.789742] loop0: detected capacity change from 0 to 204800000 [ 8829.040238] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8829.050941] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8829.439727] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8830.794552] loop0: detected capacity change from 0 to 204800000 [ 8831.173165] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8831.182617] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8831.367825] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8832.421060] loop0: detected capacity change from 0 to 204800000 [ 8832.610877] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8832.627045] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8832.761489] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8833.830610] loop0: detected capacity change from 0 to 204800000 [ 8833.857422] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8838.169587] loop0: detected capacity change from 0 to 204800000 [ 8838.205711] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8842.500105] loop0: detected capacity change from 0 to 204800000 [ 8842.526573] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8846.902740] loop0: detected capacity change from 0 to 204800000 [ 8846.930286] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8851.459326] loop0: detected capacity change from 0 to 204800000 [ 8851.545752] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8851.584502] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8851.640905] Lustre: 269492:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8851.646313] Lustre: 269492:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8851.652566] Lustre: 269492:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8851.659388] Lustre: 269492:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8851.665610] Lustre: 269492:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8851.676579] Lustre: 269492:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8851.686313] Lustre: 269492:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8851.694235] Lustre: 269492:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8851.700630] Lustre: 269492:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8851.709441] Lustre: 269492:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8851.716537] Lustre: 269492:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8851.724448] Lustre: 269492:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8851.736474] Lustre: 269492:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8851.761209] Lustre: 269492:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8857.515331] Lustre: server umount MGS complete [ 8858.498377] loop0: detected capacity change from 0 to 204800000 [ 8858.603980] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8859.353649] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8859.409687] Lustre: lustre-MDT0000: Aborting client recovery [ 8860.121547] loop1: detected capacity change from 0 to 204800000 [ 8860.253789] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8860.398337] Lustre: 269924:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8860.418943] Lustre: 269924:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8860.439182] Lustre: 269924:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8860.454969] Lustre: 269924:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8860.482564] Lustre: 269924:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8860.493874] Lustre: 269924:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8860.504564] Lustre: 269924:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8860.532624] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8860.803820] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8860.863478] Lustre: lustre-MDT0001: Aborting client recovery [ 8861.835930] loop2: detected capacity change from 0 to 204800000 [ 8861.943186] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8862.062947] Lustre: 270003:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8862.074890] Lustre: 270003:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8862.084508] Lustre: 270003:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8862.090641] Lustre: 270003:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8862.099624] Lustre: 270003:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8862.109885] Lustre: 270003:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8862.118511] Lustre: 270003:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8862.233448] Lustre: Found index 0 for lustre-OST0000, updating log [ 8862.315174] Lustre: 270021:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8862.426934] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8862.451794] Lustre: lustre-OST0000: Aborting recovery [ 8863.458027] loop3: detected capacity change from 0 to 204800000 [ 8863.581118] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8863.696127] Lustre: 270088:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8863.706974] Lustre: 270088:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8863.712801] Lustre: 270088:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8863.732893] Lustre: 270088:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8863.753642] Lustre: 270088:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8863.778208] Lustre: 270088:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8863.796630] Lustre: 270088:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8863.827233] Lustre: Found index 1 for lustre-OST0001, updating log [ 8863.868952] Lustre: 270096:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8863.940765] Lustre: lustre-OST0001: Aborting recovery [ 8869.104106] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8869.112222] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8869.129292] LustreError: 269868:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 8869.169836] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8869.211260] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8870.120284] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8870.123531] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8871.839639] LustreError: 269835:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 3905us: rc = -2 [ 8872.358607] LustreError: 269835:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 462us: rc = -2 [ 8872.378132] LustreError: 269835:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 30 previous similar messages [ 8885.733283] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8885.734839] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8885.747457] Lustre: Skipped 3 previous similar messages [ 8885.762782] Lustre: Skipped 3 previous similar messages [ 8890.861995] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8890.874907] Lustre: Skipped 3 previous similar messages [ 8891.260391] Lustre: server umount lustre-MDT0000 complete [ 8891.996747] LustreError: 269929:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779210727 with bad export cookie 11158471551300104293 [ 8892.001493] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8892.009059] LustreError: 269929:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8895.969466] LustreError: 270029:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8895.971337] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8895.972450] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8896.000710] LustreError: 270029:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 8896.020326] Lustre: Skipped 1 previous similar message [ 8896.993888] LustreError: 270029:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8897.031754] LustreError: 270029:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8898.582738] Lustre: server umount lustre-MDT0001 complete [ 8905.919381] Lustre: server umount lustre-OST0000 complete [ 8913.311203] Lustre: server umount lustre-OST0001 complete [ 8922.673891] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 13:12:36 (1779210756) [ 8951.903703] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_hostid [ 8962.306814] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 9001.358309] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9001.738632] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9001.811702] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9001.994765] Lustre: lustre-MDT0000: new disk, initializing [ 9002.163995] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9002.173373] Lustre: Skipped 1 previous similar message [ 9002.219614] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9007.848106] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 9019.809842] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9019.939850] Lustre: 274137:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9020.001931] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9020.005590] Lustre: Skipped 1 previous similar message [ 9020.085644] Lustre: lustre-MDT0001: new disk, initializing [ 9020.177634] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9020.216873] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9020.231967] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9024.749036] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 9032.752438] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9037.006927] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9044.071428] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9044.401386] Lustre: lustre-OST0000: new disk, initializing [ 9044.407322] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9044.422284] Lustre: 275874:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9044.554965] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9051.217527] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 9051.220773] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9051.244612] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9051.304965] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9058.717567] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9065.297520] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9078.588069] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9088.714211] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9097.807346] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9148.897212] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9148.904344] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9148.923834] Lustre: Skipped 1 previous similar message [ 9148.952396] Lustre: Skipped 4 previous similar messages [ 9154.024209] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9154.031238] Lustre: Skipped 1 previous similar message [ 9159.135132] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9159.433807] Lustre: server umount lustre-OST0000 complete [ 9163.746732] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9163.757300] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9163.780777] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9163.790272] Lustre: Skipped 2 previous similar messages [ 9169.939874] Lustre: server umount lustre-MDT0000 complete [ 9173.487421] LustreError: 274127:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779211008 with bad export cookie 11158471551300107009 [ 9173.492576] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9173.499489] LustreError: 274127:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9173.831778] Lustre: server umount lustre-MDT0001 complete [ 9181.970384] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9186.398405] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 9191.188773] Lustre: server umount MGS complete [ 9203.039128] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9203.455628] LustreError: 279760:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9203.482191] LustreError: 279773:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9203.548903] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9208.432320] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 9208.808496] LustreError: 279761:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9212.908375] LustreError: 279760:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9217.713256] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9218.017888] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 9218.031968] Lustre: Skipped 4 previous similar messages [ 9222.506314] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 9227.794277] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9231.487741] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9238.081276] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9238.432844] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9238.437375] Lustre: Skipped 1 previous similar message [ 9244.010568] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 9249.142604] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9279.458984] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9279.459714] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9279.461901] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9279.462540] LustreError: Skipped 1 previous similar message [ 9279.475833] Lustre: Skipped 2 previous similar messages [ 9283.853699] Lustre: server umount lustre-OST0000 complete [ 9289.697579] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9289.709695] Lustre: Skipped 1 previous similar message [ 9289.714748] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9293.637311] Lustre: server umount lustre-MDT0000 complete [ 9294.819943] LustreError: 279761:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9294.836712] LustreError: 279761:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9296.850154] LustreError: 279742:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779211132 with bad export cookie 11158471551300108864 [ 9296.852151] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9296.860554] LustreError: 279742:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9297.159837] Lustre: server umount lustre-MDT0001 complete [ 9306.572519] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing unload_modules_local [ 9309.088221] Key type lgssc unregistered [ 9309.347082] LNet: 283347:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9309.355851] LNetError: 283347:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9310.375747] LNet: Removed LNI 192.168.202.129@tcp [ 9311.453198] Key type .llcrypt unregistered [ 9311.456105] Key type ._llcrypt unregistered [ 9344.564672] Key type ._llcrypt registered [ 9344.565883] Key type .llcrypt registered [ 9344.620946] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_hostid [ 9357.114754] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 9358.142373] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9358.188826] alg: No test for adler32 (adler32-zlib) [ 9359.195858] Lustre: Lustre: Build Version: 2.17.53_25_ge261312 [ 9359.365518] LNet: Added LNI 192.168.202.129@tcp [8/256/0/180] [ 9361.007154] Key type lgssc registered [ 9361.817352] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9390.067078] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9390.100251] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9391.345688] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9391.373688] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9391.467809] Lustre: lustre-MDT0000: new disk, initializing [ 9391.540250] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9391.557546] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9395.258463] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 9403.971527] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9404.036892] Lustre: 287309:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9404.067565] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9404.069967] Lustre: Skipped 1 previous similar message [ 9404.106950] Lustre: lustre-MDT0001: new disk, initializing [ 9404.174649] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9404.205686] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9404.218719] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9407.468319] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 9413.841563] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9416.967671] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9423.606414] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9423.781553] Lustre: lustre-OST0000: new disk, initializing [ 9423.784837] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9423.789442] Lustre: 289051:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9423.836682] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9428.573232] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 9432.641409] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9432.647251] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9432.728061] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9434.760697] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9440.991871] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9453.843821] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9462.930495] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9470.392850] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9524.708374] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9524.719346] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9524.725033] Lustre: Skipped 1 previous similar message [ 9528.389644] Lustre: server umount lustre-OST0000 complete [ 9532.390270] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9532.397622] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9532.404888] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9532.410957] Lustre: Skipped 1 previous similar message [ 9534.947926] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9534.948401] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9534.985454] Lustre: Skipped 1 previous similar message [ 9537.599535] Lustre: server umount lustre-MDT0000 complete [ 9540.586897] LustreError: 287302:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779211375 with bad export cookie 12344976266096813665 [ 9540.593253] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9540.600389] LustreError: 287302:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9540.891785] Lustre: server umount lustre-MDT0001 complete [ 9548.387277] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9551.860709] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 9555.546675] Lustre: server umount MGS complete [ 9564.961126] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9565.442660] LustreError: 292982:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9565.471314] LustreError: 292982:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 9565.483782] LustreError: 292995:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9565.543553] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9569.396198] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 9570.816140] LustreError: 292983:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9574.947290] LustreError: 292982:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9576.535454] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9576.832839] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9580.465531] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 9585.599283] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9588.745618] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9594.857097] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9595.130929] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9600.403539] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 9605.193927] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9637.355244] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9637.362822] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9637.373623] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9641.446568] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9641.455219] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9641.469763] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9643.460856] Lustre: server umount lustre-OST0000 complete [ 9651.680600] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9651.690596] Lustre: Skipped 4 previous similar messages [ 9660.383171] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9660.538798] Lustre: server umount lustre-MDT0000 complete [ 9661.920288] LustreError: 292982:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9661.951828] LustreError: 292982:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9664.124480] LustreError: 295478:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779211499 with bad export cookie 12344976266096815618 [ 9664.125167] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9664.132762] LustreError: 295478:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9664.399688] Lustre: server umount lustre-MDT0001 complete [ 9674.362297] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing unload_modules_local [ 9676.738262] Key type lgssc unregistered [ 9677.034569] LNet: 296571:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9677.045309] LNetError: 296571:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9677.064345] LNet: Removed LNI 192.168.202.129@tcp [ 9677.619321] Key type .llcrypt unregistered [ 9677.624630] Key type ._llcrypt unregistered [ 9694.290759] Key type ._llcrypt registered [ 9694.294446] Key type .llcrypt registered [ 9694.405401] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_hostid [ 9706.340875] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 9707.480742] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9707.508166] alg: No test for adler32 (adler32-zlib) [ 9708.476336] Lustre: Lustre: Build Version: 2.17.53_25_ge261312 [ 9708.667872] LNet: Added LNI 192.168.202.129@tcp [8/256/0/180] [ 9710.311189] Key type lgssc registered [ 9711.304716] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9729.541411] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [ 9741.050829] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9741.087135] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9742.286282] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9742.310736] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9742.385958] Lustre: lustre-MDT0000: new disk, initializing [ 9742.457685] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9742.471207] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9745.294886] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [ 9752.751269] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9752.991531] Lustre: lustre-OST0000: new disk, initializing [ 9752.995790] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9753.000593] Lustre: Skipped 1 previous similar message [ 9753.004196] Lustre: 300594:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9753.073511] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9754.157300] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9754.162190] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9754.189678] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9756.984319] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [10312.162377] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10312.169321] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10316.856919] Lustre: server umount lustre-MDT0000 complete [10319.429910] LustreError: 301517:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779212154 with bad export cookie 467551247513977445 [10319.435581] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10319.435753] LustreError: 301517:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10319.589563] Lustre: server umount lustre-OST0000 complete [10337.880068] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_hostid [10345.292972] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [10373.136972] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10373.332530] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10373.355832] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10373.407567] Lustre: lustre-MDT0000: new disk, initializing [10373.461510] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10373.480826] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10376.363844] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [10383.553718] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10383.625498] Lustre: 304848:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10383.659227] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10383.661852] Lustre: Skipped 1 previous similar message [10383.720966] Lustre: lustre-MDT0001: new disk, initializing [10383.787334] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10383.818855] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10383.827404] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10386.500917] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [10392.331671] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10395.107862] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10399.970440] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10400.131737] Lustre: lustre-OST0000: new disk, initializing [10400.134775] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10400.139174] Lustre: 306586:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10400.212469] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10404.632532] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [10409.003593] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10409.021507] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10409.122731] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10410.079914] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10412.811283] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10412.944717] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10416.344504] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10416.509492] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10419.171188] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10419.174725] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10419.177111] Lustre: Skipped 1 previous similar message [10424.004497] Lustre: server umount lustre-OST0000 complete [10429.410130] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10429.418809] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10429.421264] Lustre: Skipped 1 previous similar message [10429.428694] Lustre: Skipped 2 previous similar messages [10432.570741] Lustre: server umount lustre-MDT0000 complete [10434.975994] LustreError: 304839:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779212270 with bad export cookie 467551247516918236 [10434.977140] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10434.985680] LustreError: 304839:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10435.257384] Lustre: server umount lustre-MDT0001 complete [10437.627312] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10438.657162] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 13:37:53 (1779212273) [10443.181235] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10443.449052] LustreError: 308750:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10443.461480] LustreError: 308750:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10443.496560] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10446.091058] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [10448.865110] LustreError: 308751:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10451.683908] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10451.894389] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10454.533862] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [10458.574767] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10460.946316] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10465.316348] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10465.525596] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10469.692692] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [10473.169750] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10477.738374] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10477.864129] Lustre: lustre-OST0001: new disk, initializing [10477.866310] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10477.869309] Lustre: 311242:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10481.286717] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [10482.744251] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10482.749491] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10482.773346] LustreError: 311686:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10483.810861] LustreError: 311691:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10483.816325] LustreError: 311691:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10484.836753] LustreError: 311686:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10484.849135] LustreError: 311686:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10486.884683] LustreError: 311691:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10486.891150] LustreError: 311691:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10487.650862] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10487.781945] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10490.119673] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffffa0ba19331800.ost_server_uuid 50 [10490.966114] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffffa0ba19331800.ost_server_uuid in FULL state after 0 sec [10490.978801] LustreError: 311691:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10490.985794] LustreError: 311691:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [10493.601567] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffffa0ba19331800.ost_server_uuid 50 [10494.847480] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffffa0ba19331800.ost_server_uuid in FULL state after 0 sec [10496.114045] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10510.397942] Lustre: lustre-OST0001: Not available for connect from 192.168.202.29@tcp (stopping) [10511.841946] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10511.846617] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10511.847672] LustreError: Skipped 1 previous similar message [10511.859461] Lustre: Skipped 1 previous similar message [10514.383284] Lustre: server umount lustre-OST0001 complete [10515.503213] LustreError: 310342:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.202.29@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10518.501248] LustreError: 311687:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10518.503181] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10518.511344] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10518.511353] Lustre: Skipped 4 previous similar messages [10518.520823] LustreError: 311687:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10518.532515] Lustre: Skipped 1 previous similar message [10518.685710] Lustre: server umount lustre-OST0000 complete [10523.617421] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10526.782940] Lustre: server umount lustre-MDT0000 complete [10528.737407] LustreError: 308746:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10528.751456] LustreError: 308746:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10528.772340] LustreError: 310364:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779212363 with bad export cookie 467551247516919118 [10528.775968] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10528.780387] LustreError: 310364:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10528.954487] Lustre: server umount lustre-MDT0001 complete [10533.692504] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing unload_modules_local [10535.049863] Key type lgssc unregistered [10535.235591] LNet: 313682:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10535.242594] LNetError: 313682:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [10535.252576] LNet: Removed LNI 192.168.202.129@tcp [10535.676268] Key type .llcrypt unregistered [10535.678418] Key type ._llcrypt unregistered [10549.053180] Key type ._llcrypt registered [10549.054288] Key type .llcrypt registered [10549.097821] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [10549.569336] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10549.583341] alg: No test for adler32 (adler32-zlib) [10550.480025] Lustre: Lustre: Build Version: 2.17.53_25_ge261312 [10550.574954] LNet: Added LNI 192.168.202.129@tcp [8/256/0/180] [10552.167208] Key type lgssc registered [10552.647981] Lustre: Echo OBD driver; http://www.lustre.org/ [10556.152983] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10556.164249] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10557.510246] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10559.129818] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [10561.467979] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10564.396985] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10564.606126] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10566.403093] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [10570.631913] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10570.826901] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10573.932751] LustreError: 316666:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10573.943071] LustreError: 316666:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10574.034643] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [10576.815371] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10579.429559] LustreError: 316666:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10580.366088] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10580.489309] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10582.585079] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10582.608147] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10583.276880] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [10585.963628] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [10589.094825] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10589.202546] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10679.778062] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10679.778837] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10679.780484] LustreError: Skipped 1 previous similar message [10679.782283] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10679.785989] Lustre: Skipped 1 previous similar message [10681.156453] Lustre: server umount lustre-OST0001 complete [10684.897420] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10684.898558] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10684.899830] LustreError: 316993:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10684.901916] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10684.901922] Lustre: Skipped 1 previous similar message [10684.907596] Lustre: Skipped 1 previous similar message [10689.178825] Lustre: server umount lustre-OST0000 complete [10692.580043] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10692.586552] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10692.593834] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10695.137991] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10695.138375] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10700.256841] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10700.259087] Lustre: Skipped 2 previous similar messages [10705.375200] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10705.460471] Lustre: server umount lustre-MDT0000 complete [10707.337788] LustreError: 318426:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779212542 with bad export cookie 942359103535707385 [10707.339225] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10707.343400] LustreError: 318426:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10707.472901] Lustre: server umount lustre-MDT0001 complete [10712.194867] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing unload_modules_local [10713.504479] Key type lgssc unregistered [10713.656419] LNet: 319663:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10713.660368] LNetError: 319663:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [10713.669437] LNet: Removed LNI 192.168.202.129@tcp [10714.001719] Key type .llcrypt unregistered [10714.003380] Key type ._llcrypt unregistered [10739.427296] Key type ._llcrypt registered [10739.428520] Key type .llcrypt registered [10739.473467] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [10739.946840] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10739.983401] alg: No test for adler32 (adler32-zlib) [10740.902127] Lustre: Lustre: Build Version: 2.17.53_25_ge261312 [10741.047224] LNet: Added LNI 192.168.202.129@tcp [8/256/0/180] [10742.663167] Key type lgssc registered [10743.246163] Lustre: Echo OBD driver; http://www.lustre.org/ [10748.085631] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10748.099238] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10749.330715] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10751.382993] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [10755.688733] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10755.845803] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10757.820459] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [10759.269771] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [10762.592208] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10762.774546] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10764.835331] LustreError: 323421:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10764.842376] LustreError: 323421:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10765.466849] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [10769.560299] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10769.700981] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10772.611364] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [10774.831079] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [10774.835182] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [10778.922121] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10783.184228] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [10790.370626] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10790.381718] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10790.384533] Lustre: Skipped 3 previous similar messages [10790.393041] Lustre: Skipped 3 previous similar messages [10795.491718] LustreError: 321968:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10795.503538] LustreError: 321968:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10796.489970] Lustre: server umount lustre-MDT0000 complete [10798.442247] LustreError: 322687:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779212633 with bad export cookie 17711651615557268037 [10798.443728] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10798.449568] LustreError: 322687:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10798.596994] Lustre: server umount lustre-MDT0001 complete [10809.927730] Lustre: server umount lustre-OST0000 complete [10820.987794] Lustre: server umount lustre-OST0001 complete [10828.182707] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [10833.484402] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10833.732876] LustreError: 327154:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10833.772555] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10835.874164] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [10839.010251] LustreError: 327155:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10840.687608] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10843.317078] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [10845.131598] Lustre: 328259:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10849.150058] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10849.345546] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10849.348452] Lustre: Skipped 1 previous similar message [10851.436537] LustreError: 328614:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10852.350602] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [10856.818343] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10857.965729] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [10857.965765] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [10859.768876] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [10864.280926] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10866.421166] Lustre: 330094:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [10868.196462] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10868.198058] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10868.201924] Lustre: Skipped 3 previous similar messages [10868.204311] Lustre: Skipped 2 previous similar messages [10873.316938] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10873.320104] Lustre: Skipped 4 previous similar messages [10874.150948] Lustre: server umount lustre-MDT0000 complete [10875.693204] LustreError: 327137:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779212710 with bad export cookie 17711651615557269724 [10875.696472] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10875.698697] LustreError: 327137:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10875.817058] Lustre: server umount lustre-MDT0001 complete [10887.698843] Lustre: server umount lustre-OST0000 complete [10899.588138] Lustre: server umount lustre-OST0001 complete [10901.532729] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [10902.257940] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [10903.041111] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 13:45:37 (1779212737) [10911.694777] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [10915.155415] loop0: detected capacity change from 0 to 1099511627776 [10915.328884] SGI XFS with ACLs, security attributes, quota, no debug enabled [10915.332521] *** VALIDATE xfs *** [10915.339135] XFS (loop0): Mounting V5 Filesystem [10915.350484] XFS (loop0): Ending clean mount [10915.828353] loop1: detected capacity change from 0 to 18691697672192 [10940.485429] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_hostid [10944.804068] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [10958.851815] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10958.956938] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10958.975051] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10959.014355] Lustre: lustre-MDT0000: new disk, initializing [10959.047886] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10959.050884] Lustre: Skipped 1 previous similar message [10959.057506] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10960.723944] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [10965.922124] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10965.966326] Lustre: 335178:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10965.989669] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10965.991795] Lustre: Skipped 1 previous similar message [10966.028661] Lustre: lustre-MDT0001: new disk, initializing [10966.082191] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10966.086707] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10967.553664] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [10970.985830] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10972.596790] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10975.457446] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10975.568931] Lustre: lustre-OST0000: new disk, initializing [10975.571586] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10975.575333] Lustre: 336919:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10976.658129] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10976.662878] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10976.715400] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10978.028843] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [10981.741250] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10983.349636] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10983.443549] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10985.297942] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10985.394895] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10986.978323] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10986.979197] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10986.979403] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10986.982388] Lustre: Skipped 1 previous similar message [10986.984037] LustreError: Skipped 1 previous similar message [10992.097627] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10992.101973] Lustre: Skipped 2 previous similar messages [10992.363272] Lustre: server umount lustre-OST0000 complete [10996.705063] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10996.708543] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11000.102626] Lustre: server umount lustre-MDT0000 complete [11001.683339] LustreError: 335168:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779212836 with bad export cookie 17711651615557271271 [11001.687216] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11001.689706] LustreError: 335168:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11001.826378] Lustre: server umount lustre-MDT0001 complete [11005.306399] XFS (loop0): Unmounting Filesystem [11007.378679] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 13:47:22 (1779212842) [11010.422428] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11010.639718] LustreError: 339323:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11010.672187] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11010.674875] Lustre: Skipped 2 previous similar messages [11012.286331] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [11015.921399] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11017.680702] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [11019.837543] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11021.235366] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11024.149155] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11026.614189] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [11028.931711] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11045.856462] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11045.858338] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11045.861583] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11045.865690] Lustre: Skipped 2 previous similar messages [11045.871702] Lustre: Skipped 4 previous similar messages [11049.072172] Lustre: server umount lustre-OST0000 complete [11065.311124] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11065.380204] Lustre: server umount lustre-MDT0000 complete [11066.336024] LustreError: 339323:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11066.342528] LustreError: 339323:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [11066.707359] LustreError: 339304:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779212901 with bad export cookie 17711651615557272153 [11066.707653] LustreError: MGC192.168.202.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11066.711466] LustreError: 339304:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11066.846489] Lustre: server umount lustre-MDT0001 complete [11071.060446] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing unload_modules_local [11072.339862] Key type lgssc unregistered [11072.477647] LNet: 342616:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11072.480891] LNetError: 342616:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [11072.488352] LNet: Removed LNI 192.168.202.129@tcp [11072.864187] Key type .llcrypt unregistered [11072.866191] Key type ._llcrypt unregistered [11084.420780] Key type ._llcrypt registered [11084.422184] Key type .llcrypt registered [11084.458209] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing load_modules_local [11084.773501] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11084.781102] alg: No test for adler32 (adler32-zlib) [11085.669269] Lustre: Lustre: Build Version: 2.17.53_25_ge261312 [11085.766640] LNet: Added LNI 192.168.202.129@tcp [8/256/0/180] [11087.367178] Key type lgssc registered [11087.795488] Lustre: Echo OBD driver; http://www.lustre.org/ [11091.112337] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11091.122917] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11092.326451] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11093.920579] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [11097.307844] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11097.503304] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11099.146485] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [11101.533215] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11102.888100] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11105.417094] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11105.549885] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11107.775661] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [11109.896389] Lustre: DEBUG MARKER: oleg229-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11125.814690] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.29@tcp (stopping) [11127.267409] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11127.268444] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11127.272412] Lustre: Skipped 1 previous similar message [11130.929877] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.29@tcp (stopping) [11130.933476] Lustre: Skipped 1 previous similar message [11136.046327] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.29@tcp (stopping) [11136.050129] Lustre: Skipped 2 previous similar messages [11138.015139] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11138.079335] Lustre: server umount lustre-MDT0001 complete [11141.165155] LustreError: 344193:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.29@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11141.174175] LustreError: 344193:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11142.457686] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11142.517907] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11142.520282] Lustre: Found index 1 for lustre-MDT0001, updating log [11142.539841] Lustre: 344185:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11142.625972] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [11142.630150] Lustre: Skipped 3 previous similar messages [11142.683539] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11144.300557] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [11147.756697] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11149.286486] LustreError: 347538:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11149.290629] LustreError: 347538:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11149.293786] LustreError: 344190:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11149.313809] LustreError: 347545:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.129@tcp: cfg command failed: rc = -17 [11149.316776] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11149.316776] [11448.373134] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.29@tcp (stopping) [11449.825930] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11449.826663] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11449.835172] Lustre: Skipped 1 previous similar message [11461.087068] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11461.144926] Lustre: server umount lustre-MDT0001 complete [11463.726961] LustreError: 345408:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.29@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11465.181872] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11465.185764] LustreError: 344195:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11465.195019] LustreError: 344195:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [11465.230792] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11465.233195] Lustre: Found index 1 for lustre-MDT0001, updating log [11465.249122] Lustre: 344187:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11465.362112] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11466.824586] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [11470.821325] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11474.921221] LustreError: 359753:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11474.924214] LustreError: 359753:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11474.928139] LustreError: 344190:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11474.930657] LustreError: 344190:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11474.952401] LustreError: 359756:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [11474.960081] LustreError: 359756:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [11474.963832] LustreError: 359756:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.129@tcp: cfg command failed: rc = -17 [11474.968131] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.129@tcp [11474.968131] [11770.940672] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.29@tcp (stopping) [11770.943241] Lustre: Skipped 8 previous similar messages [11772.897023] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11772.897596] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11772.900673] Lustre: Skipped 1 previous similar message [11775.769766] Lustre: server umount lustre-MDT0001 complete [11776.044576] LustreError: 346345:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.29@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11779.549458] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11779.611163] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11779.613808] Lustre: Found index 1 for lustre-MDT0001, updating log [11779.633044] Lustre: 344188:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11779.788840] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [11781.305028] Lustre: DEBUG MARKER: oleg229-server.virtnet: executing set_default_debug -1 all [11785.190573] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11787.750400] LustreError: 371888:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11787.753623] LustreError: 371888:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11787.757667] LustreError: 344190:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11787.760289] LustreError: 344190:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11787.785229] LustreError: 371891:0:(lod_lov.c:212:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [11787.792245] LustreError: 371891:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.129@tcp: cfg command failed: rc = -114 [11787.797723] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [11787.797723]