[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 557160296 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.003147] x2apic enabled [ 0.004010] Switched APIC routing to physical x2apic. [ 0.005015] kvm-guest: setup PV IPIs [ 0.007807] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008023] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009014] pid_max: default: 32768 minimum: 301 [ 0.010146] LSM: Security Framework initializing [ 0.011066] Yama: becoming mindful. [ 0.012045] SELinux: Initializing. [ 0.013081] *** VALIDATE selinux *** [ 0.021504] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026666] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.027156] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028125] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029143] *** VALIDATE tmpfs *** [ 0.030600] *** VALIDATE proc *** [ 0.031277] *** VALIDATE cgroup *** [ 0.032010] *** VALIDATE cgroup2 *** [ 0.033273] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.034165] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.035010] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.036031] Spectre V2 : User space: Vulnerable [ 0.037010] Speculative Store Bypass: Vulnerable [ 0.040204] debug: unmapping init [mem 0xffffffff8ac59000-0xffffffff8ac60fff] [ 0.042252] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.043714] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.044027] ... version: 2 [ 0.045029] ... bit width: 48 [ 0.046015] ... generic registers: 4 [ 0.047017] ... value mask: 0000ffffffffffff [ 0.048016] ... max period: 00007fffffffffff [ 0.049016] ... fixed-purpose events: 3 [ 0.050015] ... event mask: 000000070000000f [ 0.051491] rcu: Hierarchical SRCU implementation. [ 0.053625] smp: Bringing up secondary CPUs ... [ 0.054898] x86: Booting SMP configuration: [ 0.055029] .... node #0, CPUs: #1 #2 #3 [ 0.058452] smp: Brought up 1 node, 4 CPUs [ 0.060014] smpboot: Max logical packages: 1 [ 0.061019] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.137941] node 0 deferred pages initialised in 74ms [ 0.141263] devtmpfs: initialized [ 0.142190] x86/mm: Memory block size: 128MB [ 0.145171] gcov: version magic: 0x41383552 [ 0.147295] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.148090] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.149432] pinctrl core: initialized pinctrl subsystem [ 0.150220] [ 0.150890] ************************************************************* [ 0.151013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.152142] ** ** [ 0.153016] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.154013] ** ** [ 0.155019] ** This means that this kernel is built to expose internal ** [ 0.156029] ** IOMMU data structures, which may compromise security on ** [ 0.157016] ** your system. ** [ 0.158014] ** ** [ 0.159016] ** If you see this message and you are not debugging the ** [ 0.160014] ** kernel, report this immediately to your vendor! ** [ 0.161014] ** ** [ 0.162022] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.163014] ************************************************************* [ 0.164804] NET: Registered protocol family 16 [ 0.165505] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.166067] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.167068] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.169035] cpuidle: using governor menu [ 0.170665] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.172569] PCI: Using configuration type 1 for base access [ 0.174098] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.183082] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.184035] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.185189] cryptd: max_cpu_qlen set to 1000 [ 0.188226] ACPI: Added _OSI(Module Device) [ 0.189011] ACPI: Added _OSI(Processor Device) [ 0.190008] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.191009] ACPI: Added _OSI(Processor Aggregator Device) [ 0.194226] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.197471] ACPI: Interpreter enabled [ 0.198048] ACPI: PM: (supports S0 S3 S4 S5) [ 0.200013] ACPI: Using IOAPIC for interrupt routing [ 0.201090] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.204362] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.213839] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.216042] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.219016] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.222079] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.226117] acpiphp: Slot [2] registered [ 0.227076] acpiphp: Slot [5] registered [ 0.227977] acpiphp: Slot [6] registered [ 0.229080] acpiphp: Slot [7] registered [ 0.230106] acpiphp: Slot [8] registered [ 0.231027] acpiphp: Slot [9] registered [ 0.231936] acpiphp: Slot [10] registered [ 0.233099] acpiphp: Slot [3] registered [ 0.233949] acpiphp: Slot [4] registered [ 0.235053] acpiphp: Slot [11] registered [ 0.235978] acpiphp: Slot [12] registered [ 0.237054] acpiphp: Slot [13] registered [ 0.237907] acpiphp: Slot [14] registered [ 0.238047] acpiphp: Slot [15] registered [ 0.238889] acpiphp: Slot [16] registered [ 0.240106] acpiphp: Slot [17] registered [ 0.240920] acpiphp: Slot [18] registered [ 0.241051] acpiphp: Slot [19] registered [ 0.241933] acpiphp: Slot [20] registered [ 0.243208] acpiphp: Slot [21] registered [ 0.245098] acpiphp: Slot [22] registered [ 0.245946] acpiphp: Slot [23] registered [ 0.246057] acpiphp: Slot [24] registered [ 0.246971] acpiphp: Slot [25] registered [ 0.248060] acpiphp: Slot [26] registered [ 0.248979] acpiphp: Slot [27] registered [ 0.250059] acpiphp: Slot [28] registered [ 0.251015] acpiphp: Slot [29] registered [ 0.251909] acpiphp: Slot [30] registered [ 0.253057] acpiphp: Slot [31] registered [ 0.253929] PCI host bridge to bus 0000:00 [ 0.254012] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.256015] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.257013] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.258016] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.260012] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.261014] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.263129] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.264757] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.266877] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.273014] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.277038] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.279020] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.281015] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.283013] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.287132] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.289868] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.293052] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.296813] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.302014] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.314018] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.319014] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.325413] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.330018] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.336023] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.351016] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.360506] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.368016] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.373018] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.385015] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.393000] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.398017] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.404019] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.423027] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.443885] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.452020] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.458021] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.472024] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.482926] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.489017] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.497023] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.520022] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.535380] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.542017] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.548023] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.567014] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.577024] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.579525] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.582392] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.584360] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.587256] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.590151] iommu: Default domain type: Passthrough [ 0.591329] SCSI subsystem initialized [ 0.592117] ACPI: bus type USB registered [ 0.593074] usbcore: registered new interface driver usbfs [ 0.595052] usbcore: registered new interface driver hub [ 0.596101] usbcore: registered new device driver usb [ 0.598336] pps_core: LinuxPPS API ver. 1 registered [ 0.600012] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.603103] PTP clock support registered [ 0.606076] EDAC MC: Ver: 3.0.0 [ 0.607471] PCI: Using ACPI for IRQ routing [ 0.608571] NetLabel: Initializing [ 0.609008] NetLabel: domain hash size = 128 [ 0.609990] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.610189] NetLabel: unlabeled traffic allowed by default [ 0.611119] vgaarb: loaded [ 0.612257] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.614012] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.619623] clocksource: Switched to clocksource kvm-clock [ 0.721614] VFS: Disk quotas dquot_6.6.0 [ 0.723825] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.726341] *** VALIDATE ramfs *** [ 0.727288] *** VALIDATE hugetlbfs *** [ 0.728407] pnp: PnP ACPI init [ 0.730559] pnp: PnP ACPI: found 6 devices [ 0.745206] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.748338] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.750530] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.752737] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.755285] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.757892] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.760064] NET: Registered protocol family 2 [ 0.761658] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.765598] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.770039] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.775205] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.778582] TCP: Hash tables configured (established 65536 bind 65536) [ 0.781240] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.783916] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.786623] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.788645] NET: Registered protocol family 1 [ 0.790565] RPC: Registered named UNIX socket transport module. [ 0.792279] RPC: Registered udp transport module. [ 0.793494] RPC: Registered tcp transport module. [ 0.794677] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.796570] NET: Registered protocol family 44 [ 0.798207] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.800342] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.801853] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.803527] PCI: CLS 0 bytes, default 64 [ 0.805212] Unpacking initramfs... [ 2.206716] debug: unmapping init [mem 0xffff93573cc54000-0xffff93573ffbffff] [ 2.210515] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.212784] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.215679] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.697425] Initialise system trusted keyrings [ 2.699250] Key type blacklist registered [ 2.700915] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.709098] zbud: loaded [ 2.711717] *** VALIDATE nfs *** [ 2.712966] *** VALIDATE nfs4 *** [ 2.714252] pstore: using deflate compression [ 2.717086] Platform Keyring initialized [ 2.814445] NET: Registered protocol family 38 [ 2.815816] Key type asymmetric registered [ 2.816873] Asymmetric key parser 'x509' registered [ 2.818216] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.820279] io scheduler mq-deadline registered [ 2.821601] io scheduler kyber registered [ 2.823206] io scheduler bfq registered [ 2.824765] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.826922] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.828992] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.831210] ACPI: Power Button [PWRF] [ 2.836138] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.842901] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.859434] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 2.867737] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 2.878527] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.904921] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.931027] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 2.936103] Non-volatile memory driver v1.3 [ 2.937551] Linux agpgart interface v0.103 [ 2.966271] virtio_blk virtio1: [vda] 146648 512-byte logical blocks (75.1 MB/71.6 MiB) [ 2.968184] vda: detected capacity change from 0 to 75083776 [ 2.980599] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 2.983512] vdb: detected capacity change from 0 to 1073741824 [ 2.996884] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 2.999700] vdc: detected capacity change from 0 to 2621440000 [ 3.011551] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.013571] vdd: detected capacity change from 0 to 2621440000 [ 3.025512] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.028868] vde: detected capacity change from 0 to 4294967296 [ 3.043263] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.044918] vdf: detected capacity change from 0 to 4294967296 [ 3.050704] libphy: Fixed MDIO Bus: probed [ 3.061920] usbcore: registered new interface driver usbserial_generic [ 3.065171] usbserial: USB Serial support registered for generic [ 3.068097] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.073394] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.075925] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.079053] mousedev: PS/2 mouse device common for all mice [ 3.082319] rtc_cmos 00:05: RTC can wake from S4 [ 3.085475] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.087385] rtc_cmos 00:05: registered as rtc0 [ 3.093840] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.098711] intel_pstate: CPU model not supported [ 3.099638] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.107952] hid: raw HID events driver (C) Jiri Kosina [ 3.115561] usbcore: registered new interface driver usbhid [ 3.117823] usbhid: USB HID core driver [ 3.120978] drop_monitor: Initializing network drop monitor service [ 3.122934] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.123922] Initializing XFRM netlink socket [ 3.124337] NET: Registered protocol family 10 [ 3.125623] Segment Routing with IPv6 [ 3.125655] NET: Registered protocol family 17 [ 3.126463] mpls_gso: MPLS GSO support [ 3.140322] RAS: Correctable Errors collector initialized. [ 3.141909] AVX version of gcm_enc/dec engaged. [ 3.143510] AES CTR mode by8 optimization enabled [ 3.209036] sched_clock: Marking stable (3209020886, 0)->(4196736370, -987715484) [ 3.214659] registered taskstats version 1 [ 3.218426] Loading compiled-in X.509 certificates [ 3.221674] zswap: loaded using pool lzo/zbud [ 3.267377] Key type big_key registered [ 3.289913] Key type encrypted registered [ 3.291095] ima: No TPM chip found, activating TPM-bypass! [ 3.300462] ima: Allocated hash algorithm: sha1 [ 3.302406] ima: No architecture policies found [ 3.306341] evm: Initialising EVM extended attributes: [ 3.310684] evm: security.selinux [ 3.312695] evm: security.ima [ 3.314242] evm: security.capability [ 3.315429] evm: HMAC attrs: 0x1 [ 3.320705] rtc_cmos 00:05: setting system clock to 2026-09-07 09:59:06 UTC (1788775146) [ 3.338314] debug: unmapping init [mem 0xffffffff8bc03000-0xffffffff8bdfffff] [ 3.344912] debug: unmapping init [mem 0xffffffff8a982000-0xffffffff8ac58fff] [ 3.354243] Write protecting the kernel read-only data: 28672k [ 3.362180] debug: unmapping init [mem 0xffffffff89003000-0xffffffff891fffff] [ 3.369235] debug: unmapping init [mem 0xffffffff89914000-0xffffffff899fffff] [ 3.418148] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.426300] systemd[1]: Detected virtualization kvm. [ 3.427455] systemd[1]: Detected architecture x86-64. [ 3.428410] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.464015] systemd[1]: No hostname configured. [ 3.465523] systemd[1]: Set hostname to . [ 3.472546] random: systemd: uninitialized urandom read (16 bytes read) [ 3.477569] systemd[1]: Initializing machine ID from random generator. [ 3.804167] random: systemd: uninitialized urandom read (16 bytes read) [ 3.807131] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ 3.815593] random: systemd: uninitialized urandom read (16 bytes read) [ 3.817571] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 3.830867] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket. Starting Apply Kernel Variables... Starting Setup Virtual Console... [ OK ] Reached target Timers. [ OK ] Reached target Initrd Root Device. [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Swap. Starting Journal Service... [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. Starting Create Volatile Files and Directories... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ 5.479050] hrtimer: interrupt took 8039083 ns [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 6.264815] device-mapper: uevent: version 1.0.3 [ 6.275875] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 8.694620] virtio_net virtio0 ens2: renamed from eth0 [ 8.736649] scsi host0: ata_piix [ 8.750775] scsi host1: ata_piix [ 8.751605] random: fast init done [ 8.760852] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 8.775401] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 13.847950] random: crng init done [ 13.849257] random: 7 urandom warning(s) missed due to ratelimiting [ 15.740552] dracut-initqueue[584]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 17.831539] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Swap. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 20.559779] printk: systemd: 26 output lines suppressed due to ratelimiting [ 21.316183] SELinux: Disabled at runtime. [ 21.450235] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 21.463757] systemd[1]: Detected virtualization kvm. [ 21.469643] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 23.735131] systemd[1]: initrd-switch-root.service: Succeeded. [ 23.759683] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 23.792355] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 23.812211] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 23.826692] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 23.874175] systemd[1]: Starting Journal Service... Starting Journal Service... [ 23.937102] systemd[1]: Mounting Kernel Debug File System... Mounting Kernel Debug File System... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Listening on udev Control Socket. [ OK ] Created slice User and Session Slice. Starting Apply Kernel Variables... [ OK ] Listening on udev Kernel Socket. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Created slice system-serial\x2dgetty.slice. Mounting Huge Pages File System... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. Starting udev Coldplug all Devices... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Reached target rpc_pipefs.target. [ OK ] Reached target Slices. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Listening on initctl Compatibility Named Pipe. Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on Process Core Dump Socket. [ OK ] Created slice system-getty.slice. [ 24.612534] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. Starting Remount Root and Kernel File Systems... [ OK ] Stopped target Initrd File Systems. [ OK ] Created slice system-sshd\x2dkeygen.slice. Mounting POSIX Message Queue File System... [ OK ] Started Journal Service. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Huge Pages File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted POSIX Message Queue File System. Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started udev Coldplug all Devices. [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 26.060302] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 27.651964] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 27.667098] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 28.160996] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 28.253393] EDAC sbridge: Ver: 1.1.2 [ 32.453642] Key type dns_resolver registered [* ] A start job is running for Configur…-only root support (9s / no limit) [** ] A start job is running for Configur…-only root support (9s / no limit)[ 33.601691] NFS: Registering the id_resolver key type [ 33.613446] Key type id_resolver registered [ 33.619055] Key type id_legacy registered [*** ] A start job is running for Configur…only root support (10s / no limit) [ *** ] A start job is running for Configur…only root support (10s / no limit) [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Login Service... [ OK ] Started D-Bus System Message Bus. [ OK ] Started Daily Cleanup of Temporary Directories. Starting Restore /run/initramfs on shutdown... [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started irqbalance daemon. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Network Manager... [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Started Login Service. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... Starting Hostname Service... [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting System Logging Service... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg248-server login: [ 106.702020] libcfs: loading out-of-tree module taints kernel. [ 106.761700] Key type ._llcrypt registered [ 106.766375] Key type .llcrypt registered [ 106.889329] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [ 129.973811] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 131.477929] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 131.495594] alg: No test for adler32 (adler32-zlib) [ 132.943802] Lustre: Lustre: Build Version: 2.17.57_103_g80273de [ 134.162984] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 136.216076] Key type lgssc registered [ 139.227908] Lustre: Echo OBD driver; http://www.lustre.org/ [ 164.712696] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 215.926565] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 233.660892] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 233.691454] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 235.180247] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 235.213600] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 235.334275] Lustre: lustre-MDT0000: new disk, initializing [ 235.445898] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 235.490766] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 241.636040] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 256.659642] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 256.758807] Lustre: 6534:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 256.847590] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 256.854781] Lustre: Skipped 1 previous similar message [ 257.098511] Lustre: lustre-MDT0001: new disk, initializing [ 257.253745] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 257.325661] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 257.350434] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 263.922664] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 269.787642] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 280.717446] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 280.955527] Lustre: lustre-OST0000: new disk, initializing [ 280.957841] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 280.960581] Lustre: 8472:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 281.042047] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 281.836640] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 281.846760] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 281.926623] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 288.990803] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 305.390445] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 305.459709] Lustre: lustre-OST0001: new disk, initializing [ 305.462602] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 305.466282] Lustre: 9546:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 305.518588] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 310.878796] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 310.897828] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 311.022121] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 312.856756] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 325.566525] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 338.560206] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 345.839748] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing check_logdir /tmp/testlogs/ [ 351.933451] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing yml_node [ 356.851849] Lustre: DEBUG MARKER: Client: 2.17.57.103 [ 359.681890] Lustre: DEBUG MARKER: MDS: 2.17.57.103 [ 362.365611] Lustre: DEBUG MARKER: OSS: 2.17.57.103 [ 364.144852] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Mon Sep 7 06:05:05 EDT 2026 [ 383.874933] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 385.835324] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 403.424574] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 403.427070] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 403.453501] Lustre: Skipped 3 previous similar messages [ 403.456706] Lustre: Skipped 2 previous similar messages [ 408.417370] Lustre: server umount lustre-MDT0000 complete [ 413.549931] LustreError: 6528:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788775556 with bad export cookie 2193175898201545004 [ 413.564672] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 413.574204] LustreError: 6528:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 413.664554] LustreError: 8480:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 413.668958] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 413.671317] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 413.671326] Lustre: Skipped 1 previous similar message [ 413.695331] LustreError: 8480:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 413.733244] Lustre: Skipped 1 previous similar message [ 418.786504] LustreError: 6540:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 418.789591] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 418.822611] Lustre: Skipped 2 previous similar messages [ 420.341267] Lustre: server umount lustre-MDT0001 complete [ 426.727871] Lustre: server umount lustre-OST0000 complete [ 431.538737] Lustre: server umount lustre-OST0001 complete [ 439.689634] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [ 450.769650] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 488.972144] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 489.292376] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 489.337466] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 489.406445] Lustre: lustre-MDT0000: new disk, initializing [ 489.528897] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 489.547347] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 494.748779] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 506.226419] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 506.346437] Lustre: 15099:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 506.397876] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 506.402315] Lustre: Skipped 1 previous similar message [ 506.474648] Lustre: lustre-MDT0001: new disk, initializing [ 506.638748] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 506.738704] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 506.765370] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 512.922798] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 523.997805] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 530.050537] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 536.929409] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 537.188367] Lustre: lustre-OST0000: new disk, initializing [ 537.192534] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 537.200874] Lustre: 17069:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 537.290185] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 538.428753] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 538.452775] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 538.687314] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 544.050901] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 554.922630] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 559.845574] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 560.196792] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 566.045363] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 566.692728] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 574.432801] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 574.440765] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 574.450379] LustreError: Skipped 1 previous similar message [ 574.456724] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 574.508435] Lustre: Skipped 1 previous similar message [ 575.994333] Lustre: server umount lustre-OST0000 complete [ 583.656170] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 583.665797] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 583.694511] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 583.703267] Lustre: Skipped 1 previous similar message [ 587.696910] Lustre: server umount lustre-MDT0000 complete [ 589.795785] LustreError: 15110:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 589.806231] LustreError: 15110:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 591.440540] LustreError: 15091:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788775734 with bad export cookie 2193175898201546537 [ 591.441886] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 591.446838] LustreError: 15091:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 591.807946] Lustre: server umount lustre-MDT0001 complete [ 597.026064] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 06:08:58 (1788775738) [ 604.889056] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 605.461475] LustreError: 19333:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 605.528834] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 610.298567] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 610.787713] LustreError: 19334:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 620.002534] LustreError: 19334:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 620.017465] LustreError: 19334:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 620.143318] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 625.883627] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 633.541197] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 640.045246] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 648.985418] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 656.259571] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 665.778406] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 669.757373] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 676.838786] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 676.843170] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 676.862894] Lustre: Skipped 1 previous similar message [ 676.880820] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 676.887895] Lustre: Skipped 2 previous similar messages [ 683.260609] Lustre: server umount lustre-MDT0000 complete [ 685.538543] LustreError: 20061:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 685.567166] LustreError: 20061:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 687.484241] LustreError: 21863:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788775830 with bad export cookie 2193175898201547419 [ 687.493124] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 687.496109] LustreError: 21863:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 688.125937] Lustre: server umount lustre-MDT0001 complete [ 702.543128] Lustre: server umount lustre-OST0000 complete [ 725.283574] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 737.424670] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 737.800970] LustreError: 23716:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 737.863841] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 737.873254] Lustre: Skipped 2 previous similar messages [ 742.064553] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 751.455337] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 756.782789] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 759.992633] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 767.169439] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 774.998417] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 784.844362] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 785.055876] Lustre: lustre-OST0001: new disk, initializing [ 785.058972] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 785.064278] Lustre: 26029:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 786.797706] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 786.812404] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 786.932736] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 792.565600] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 802.639677] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 806.958279] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 822.758217] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 822.761324] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 822.780715] Lustre: Skipped 5 previous similar messages [ 822.784674] Lustre: Skipped 5 previous similar messages [ 826.333410] Lustre: server umount lustre-MDT0000 complete [ 827.882478] LustreError: 23711:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 827.916264] LustreError: 23711:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 830.067099] LustreError: 26794:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788775973 with bad export cookie 2193175898201548266 [ 830.076845] LustreError: 26794:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 830.083284] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 830.529454] Lustre: server umount lustre-MDT0001 complete [ 844.293494] Lustre: server umount lustre-OST0000 complete [ 858.789307] Lustre: server umount lustre-OST0001 complete [ 877.401965] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 888.591233] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 889.148087] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 889.157620] Lustre: Skipped 3 previous similar messages [ 893.982279] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 904.073893] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 909.445247] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 912.625656] Lustre: 30216:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 919.914699] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 927.105892] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 935.536834] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 942.161445] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 950.515945] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 954.310921] Lustre: 32094:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 960.995555] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 961.001407] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 961.017648] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 961.023675] Lustre: Skipped 1 previous similar message [ 967.278165] Lustre: server umount lustre-MDT0000 complete [ 971.426886] LustreError: 29038:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788776114 with bad export cookie 2193175898201549554 [ 971.454383] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 971.459214] LustreError: 29038:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 971.818058] LustreError: 29053:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 971.838626] LustreError: 29053:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [ 972.168789] Lustre: server umount lustre-MDT0001 complete [ 978.904835] Lustre: server umount lustre-OST0000 complete [ 983.310846] Lustre: server umount lustre-OST0001 complete [ 993.419086] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 06:15:34 (1788776134) [ 1024.145845] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 1037.339867] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1043.885908] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1053.716886] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1060.297345] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1063.780975] Lustre: 36047:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1071.873992] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1079.915209] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1090.105856] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1097.365871] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1104.921129] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1108.968551] Lustre: 37924:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1115.623194] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1115.633862] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1115.646086] Lustre: Skipped 5 previous similar messages [ 1115.659144] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1115.663884] Lustre: Skipped 8 previous similar messages [ 1120.902332] Lustre: server umount lustre-MDT0000 complete [ 1124.763090] LustreError: 37927:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788776267 with bad export cookie 2193175898201550667 [ 1124.784853] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1124.785545] LustreError: 37927:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1125.063109] Lustre: server umount lustre-MDT0001 complete [ 1139.455512] Lustre: server umount lustre-OST0000 complete [ 1142.175799] Lustre: 3664:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788776269/real 1788776269] req@ffff9357ac07f800 x1875666825206400/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788776285 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1144.472864] Lustre: server umount lustre-OST0001 complete [ 1153.350823] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 06:18:14 (1788776294) [ 1172.154667] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 1183.763791] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1184.446602] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1184.464112] Lustre: Skipped 7 previous similar messages [ 1189.418705] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1200.409097] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1206.203344] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1209.347889] Lustre: 41216:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1216.421876] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1224.264951] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1229.234558] LustreError: 41569:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1229.262038] LustreError: 41569:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 19 previous similar messages [ 1233.049342] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1240.422405] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1249.087279] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1253.425768] Lustre: 43095:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1254.549760] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1254.552645] Lustre: Skipped 1 previous similar message [ 1257.440738] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1257.447208] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1257.456131] Lustre: Skipped 4 previous similar messages [ 1257.461117] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1257.465539] Lustre: Skipped 4 previous similar messages [ 1263.697220] Lustre: server umount lustre-MDT0000 complete [ 1267.658937] LustreError: 40047:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788776410 with bad export cookie 2193175898201551773 [ 1267.663461] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1267.667173] LustreError: 40047:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1274.049629] Lustre: server umount lustre-MDT0001 complete [ 1282.489596] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1292.769299] LustreError: 3662:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff9357ac07dc00 x1875666825259264/t0(0) o250->MGC192.168.202.148@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1298.810244] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1302.528055] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 1308.340332] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1314.205647] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1314.325528] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1314.330636] Lustre: Skipped 1 previous similar message [ 1323.020614] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1325.002530] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1329.959409] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1331.590899] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1339.368098] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1342.546641] Lustre: server umount lustre-MDT0000 complete [ 1346.802845] LustreError: 40046:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788776489 with bad export cookie 2193175898201552900 [ 1346.804532] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1346.817558] LustreError: 40046:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1347.133945] Lustre: server umount lustre-MDT0001 complete [ 1361.131995] Lustre: server umount lustre-OST0000 complete [ 1374.042403] Lustre: server umount lustre-OST0001 complete [ 1384.544483] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 06:22:05 (1788776525) [ 1402.173969] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 1413.183056] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1418.933600] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1427.406701] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1432.527418] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1435.593757] Lustre: 48861:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1442.468654] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1449.548547] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1458.119697] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1465.664484] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1473.229931] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1477.856181] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1494.503524] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1494.512868] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1494.532309] Lustre: Skipped 9 previous similar messages [ 1506.271227] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1506.575746] Lustre: server umount lustre-MDT0000 complete [ 1511.642037] LustreError: 47691:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788776654 with bad export cookie 2193175898201553698 [ 1511.643108] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1511.662611] LustreError: 47691:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1512.124903] Lustre: server umount lustre-MDT0001 complete [ 1525.968320] Lustre: server umount lustre-OST0000 complete [ 1540.326501] Lustre: server umount lustre-OST0001 complete [ 1549.597945] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 06:24:51 (1788776691) [ 1552.164829] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1553.927390] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 06:24:55 (1788776695) [ 1564.987255] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [ 1574.355596] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 1597.996733] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1598.251631] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1598.396422] Lustre: lustre-MDT0000: new disk, initializing [ 1598.527408] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1603.946505] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1613.360203] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1619.252251] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1619.449714] Lustre: lustre-OST0000: new disk, initializing [ 1619.453823] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1619.458465] Lustre: Skipped 1 previous similar message [ 1619.463464] Lustre: 54941:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1621.322162] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1621.334828] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1621.372662] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1625.540494] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1634.275372] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1636.451373] Lustre: DEBUG MARKER: create test files [ 1667.556851] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1667.559403] Lustre: Skipped 27 previous similar messages [ 1671.498693] Lustre: server umount lustre-OST0000 complete [ 1675.872308] Lustre: server umount lustre-MDT0000 complete [ 1689.440165] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 1692.716473] Key type lgssc unregistered [ 1693.153088] LNet: 56572:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1693.172036] LNetError: 56572:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1693.190280] LNet: Removed LNI 192.168.202.148@tcp [ 1694.076333] Key type .llcrypt unregistered [ 1694.078855] Key type ._llcrypt unregistered [ 1720.810587] Key type ._llcrypt registered [ 1720.817124] Key type .llcrypt registered [ 1720.938096] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 1721.846436] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1721.990494] alg: No test for adler32 (adler32-zlib) [ 1723.098300] Lustre: Lustre: Build Version: 2.17.57_103_g80273de [ 1723.357837] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 1725.023846] Key type lgssc registered [ 1726.142894] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1733.130745] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1733.157313] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1734.490500] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1739.015505] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1745.878406] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1751.449795] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1751.804583] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1758.968863] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:98 to 0x240000400:129) [ 1759.180534] Lustre: 59281:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1759.192077] Lustre: 59281:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1759.201106] Lustre: 59281:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1759.218061] Lustre: 59281:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1759.241084] Lustre: 59281:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1759.252832] Lustre: 59281:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1759.402769] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1769.775973] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1775.904733] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1783.043408] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1815.522882] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1815.537470] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1815.570406] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1817.679984] Lustre: server umount lustre-OST0000 complete [ 1821.813463] Lustre: server umount lustre-MDT0000 complete [ 1832.378924] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 1835.467639] Key type lgssc unregistered [ 1835.698779] LNet: 60917:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1835.705469] LNetError: 60917:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1835.717185] LNet: Removed LNI 192.168.202.148@tcp [ 1836.560673] Key type .llcrypt unregistered [ 1836.562886] Key type ._llcrypt unregistered [ 1871.938510] Key type ._llcrypt registered [ 1871.943950] Key type .llcrypt registered [ 1872.058622] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 1873.457969] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1873.509370] alg: No test for adler32 (adler32-zlib) [ 1874.464428] Lustre: Lustre: Build Version: 2.17.57_103_g80273de [ 1874.848813] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 1876.496059] Key type lgssc registered [ 1877.626359] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1885.161802] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1885.173305] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1886.606573] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1891.457697] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1899.740295] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1906.181528] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1906.516200] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1911.665801] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1911.773257] Lustre: 63748:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1911.777513] Lustre: 63748:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1911.781522] Lustre: 63748:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1911.785715] Lustre: 63748:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1911.789854] Lustre: 63748:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1911.793887] Lustre: 63748:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1912.105308] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1918.849187] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1923.990589] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1937.387769] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1937.388771] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1937.457652] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1939.107894] Lustre: server umount lustre-OST0000 complete [ 1943.404506] Lustre: server umount lustre-MDT0000 complete [ 1954.941467] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 1957.430892] Key type lgssc unregistered [ 1957.824523] LNet: 65303:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1957.834015] LNetError: 65303:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1957.848434] LNet: Removed LNI 192.168.202.148@tcp [ 1959.058243] Key type .llcrypt unregistered [ 1959.060284] Key type ._llcrypt unregistered [ 1975.716315] Key type ._llcrypt registered [ 1975.726725] Key type .llcrypt registered [ 1975.879575] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [ 1990.904243] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 1991.578304] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1991.615898] alg: No test for adler32 (adler32-zlib) [ 1992.776028] Lustre: Lustre: Build Version: 2.17.57_103_g80273de [ 1993.274052] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 1995.002230] Key type lgssc registered [ 1997.449621] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2034.654097] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2034.682265] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2035.973486] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2036.006610] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2036.117908] Lustre: lustre-MDT0000: new disk, initializing [ 2036.192564] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2036.204878] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2040.720284] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2050.116087] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2050.178640] Lustre: 68453:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2050.222535] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2050.229155] Lustre: Skipped 1 previous similar message [ 2050.303975] Lustre: lustre-MDT0001: new disk, initializing [ 2050.365473] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2050.391197] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2050.400461] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2054.811712] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2064.614579] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2070.228434] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2076.675755] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2076.975722] Lustre: lustre-OST0000: new disk, initializing [ 2076.981472] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2076.989838] Lustre: 70423:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2077.074374] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2083.219928] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2084.958685] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2084.972229] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2085.025928] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2091.884557] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2096.430301] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2096.632815] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2101.043395] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2101.219258] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2105.827552] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2105.842070] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2105.860201] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2105.868660] Lustre: Skipped 1 previous similar message [ 2109.139378] Lustre: server umount lustre-OST0000 complete [ 2116.065928] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2116.071754] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2116.074536] Lustre: Skipped 2 previous similar messages [ 2116.082671] Lustre: Skipped 1 previous similar message [ 2118.304762] Lustre: server umount lustre-MDT0000 complete [ 2121.181815] LustreError: 68446:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788777264 with bad export cookie 17393430377734239594 [ 2121.183233] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2121.193618] LustreError: 68446:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2121.448995] Lustre: server umount lustre-MDT0001 complete [ 2129.764409] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 06:34:31 (1788777271) [ 2138.666803] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 2179.446718] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [ 2190.329754] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 2228.700645] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2228.991708] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2229.027262] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2229.118232] Lustre: lustre-MDT0000: new disk, initializing [ 2229.252208] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2229.300503] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2233.695374] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2244.450164] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2244.560782] Lustre: 76444:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2244.633184] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2244.637846] Lustre: Skipped 1 previous similar message [ 2244.741818] Lustre: lustre-MDT0001: new disk, initializing [ 2244.838416] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2244.895186] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2244.913502] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2249.572959] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2258.784435] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2263.913757] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2270.541126] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2270.784482] Lustre: lustre-OST0000: new disk, initializing [ 2270.788550] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2270.793798] Lustre: 78409:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2270.909316] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2277.412431] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2279.523716] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2279.533854] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2279.634308] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2285.564913] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2289.895076] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2290.044560] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2294.497475] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2294.712741] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2300.388411] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2300.392328] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2300.406660] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2300.407913] LustreError: Skipped 2 previous similar messages [ 2300.435306] Lustre: Skipped 1 previous similar message [ 2305.508974] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2305.518516] Lustre: Skipped 2 previous similar messages [ 2310.636124] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2311.135502] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2311.340295] Lustre: server umount lustre-OST0000 complete [ 2315.745234] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2315.769121] Lustre: Skipped 1 previous similar message [ 2320.865622] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2320.868211] Lustre: Skipped 4 previous similar messages [ 2321.711976] Lustre: server umount lustre-MDT0000 complete [ 2324.921895] LustreError: 78420:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788777468 with bad export cookie 17393430377734240476 [ 2324.923174] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2324.927200] LustreError: 78420:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2325.157546] Lustre: server umount lustre-MDT0001 complete [ 2333.040865] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 06:37:54 (1788777474) [ 2341.118422] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2341.827066] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2346.978575] LustreError: 80826:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2347.024899] LustreError: 80826:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 2347.229386] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2351.078736] LustreError: 80827:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2356.197601] LustreError: 80826:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2356.667702] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2360.578953] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2366.389467] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2372.246526] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2380.290399] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2380.861616] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2380.870817] Lustre: Skipped 1 previous similar message [ 2388.047232] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2395.974933] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2400.012087] Lustre: 83498:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting 7a1477cb-e4ad-4f19-a66e-e4f1016417a0 at adminstrative request [ 2401.255519] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2401.288964] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2411.495647] Lustre: MGS: Client 7a1477cb-e4ad-4f19-a66e-e4f1016417a0 (at 0@lo) reconnecting [ 2411.505580] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xf161e0c632567c4e to 0xf161e0c632567eee [ 2411.524283] Lustre: MGC192.168.202.148@tcp: Connection restored to 0@lo (at 0@lo) [ 2438.288711] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2438.548950] Lustre: lustre-OST0001: new disk, initializing [ 2438.552378] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2438.561410] Lustre: 83890:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2445.797990] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2448.000286] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2448.011473] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2448.130377] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2455.785124] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2463.715240] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2463.716355] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2463.721975] LustreError: Skipped 1 previous similar message [ 2463.722918] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2463.733708] Lustre: Skipped 1 previous similar message [ 2465.574925] Lustre: server umount lustre-OST0001 complete [ 2468.853390] LustreError: 83312:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2468.878352] LustreError: 83312:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2472.928736] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2472.947697] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2473.958685] LustreError: 82648:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2473.998763] LustreError: 82648:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2475.938952] Lustre: server umount lustre-OST0000 complete [ 2486.935835] Lustre: server umount lustre-MDT0000 complete [ 2489.314246] LustreError: 80827:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2489.333631] LustreError: 80827:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2490.534642] LustreError: 80808:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788777633 with bad export cookie 17393430377734242030 [ 2490.539214] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2490.544607] LustreError: 80808:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2491.005640] Lustre: server umount lustre-MDT0001 complete [ 2499.126880] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 06:40:40 (1788777640) [ 2500.660708] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2502.441506] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 06:40:44 (1788777644) [ 2504.191458] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2506.091638] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 06:40:47 (1788777647) [ 2507.568760] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2509.429276] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 06:40:50 (1788777650) [ 2511.103637] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2512.829700] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 06:40:54 (1788777654) [ 2514.426446] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2520.481900] loop: module loaded [ 2520.508818] loop0: detected capacity change from 0 to 4294967296 [ 2520.557069] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2522.308508] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2526.127652] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2530.275184] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2557.169120] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2557.211933] LustreError: 88015:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2557.220979] LustreError: 88015:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2557.227239] LustreError: 88015:0:(tgt_mount.c:2582:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2557.247135] LustreError: 88015:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2576.583332] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [ 2587.497969] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 2626.034747] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2626.407864] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2626.508756] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2626.701831] Lustre: lustre-MDT0000: new disk, initializing [ 2626.886245] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2626.894197] Lustre: Skipped 1 previous similar message [ 2626.927557] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2630.798617] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2642.677563] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2642.747339] Lustre: 90883:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2642.886221] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2647.051547] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2656.743562] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2663.076754] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2669.778760] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2669.974691] Lustre: 92853:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2676.548457] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2678.353708] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2678.506210] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2685.757116] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2689.745614] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2689.879772] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2693.624409] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2693.840551] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2699.240344] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2699.243445] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2699.244333] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2699.248181] Lustre: Skipped 4 previous similar messages [ 2699.267120] Lustre: Skipped 7 previous similar messages [ 2701.616570] Lustre: server umount lustre-OST0000 complete [ 2709.479235] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2709.515087] LustreError: Skipped 1 previous similar message [ 2720.223164] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2720.531468] Lustre: server umount lustre-MDT0000 complete [ 2724.702711] LustreError: 90874:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788777867 with bad export cookie 17393430377734242646 [ 2724.722400] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2724.722797] LustreError: 90874:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2725.016906] Lustre: server umount lustre-MDT0001 complete [ 2733.938630] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 06:44:35 (1788777875) [ 2741.794328] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2742.179817] LustreError: 95264:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2747.375204] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2757.205178] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2757.813637] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2757.826417] Lustre: Skipped 3 previous similar messages [ 2762.618912] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2770.377488] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2778.070594] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2785.581490] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2792.721185] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2801.215264] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2806.477744] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2806.664624] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2810.379613] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2810.599507] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2819.040863] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2819.312414] Lustre: lustre-OST0001: new disk, initializing [ 2819.315927] Lustre: Skipped 2 previous similar messages [ 2819.321083] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2819.328875] Lustre: Skipped 3 previous similar messages [ 2819.337493] Lustre: 98633:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2827.477478] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2828.042144] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2828.068533] Lustre: Skipped 2 previous similar messages [ 2828.094479] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2828.510405] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2842.047238] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2847.612366] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2847.826381] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2852.918737] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2853.113647] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2873.436365] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2901.414471] Lustre: Failing over lustre-MDT0000 [ 2901.642577] Lustre: lustre-MDT0000: Not available for connect from 192.168.202.48@tcp (stopping) [ 2901.645285] Lustre: Skipped 7 previous similar messages [ 2901.885912] Lustre: server umount lustre-MDT0000 complete [ 2905.062883] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2905.089215] Lustre: Skipped 4 previous similar messages [ 2905.090443] LustreError: 97159:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2905.126342] LustreError: 97159:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 2921.449498] Lustre: 66674:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788778048/real 1788778048] req@00000000fe71990a x1875668774436736/t0(0) o400->MGC192.168.202.148@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1788778064 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2921.476637] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2927.065932] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2927.068773] LDISKFS-fs (dm-0): recovery complete [ 2927.083651] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2931.680293] LustreError: 66673:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000e44259b0 x1875668774445824/t0(0) o250->MGC192.168.202.148@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2932.108419] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2932.338962] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2932.347750] Lustre: 97159:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2936.433709] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2937.323166] Lustre: 99017:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2937.339499] Lustre: 99017:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 2937.373789] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2937.465339] LustreError: 101487:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2937.796519] LustreError: 101487:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2938.133114] LustreError: 101487:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2938.145612] LustreError: 101487:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2938.487164] LustreError: 101487:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2938.497068] LustreError: 101487:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2938.513154] Lustre: 101487:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2938.525547] Lustre: 101487:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 2939.171048] LustreError: 101487:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2939.180786] LustreError: 101487:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2939.511142] LustreError: 101487:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2939.523370] LustreError: 101487:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2940.810104] Lustre: 101487:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2940.829787] Lustre: 101487:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 6 previous similar messages [ 2941.479577] LustreError: 101487:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2941.489460] LustreError: 101487:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2941.815839] LustreError: 101487:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2941.827358] LustreError: 101487:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2945.034417] Lustre: 101487:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2945.053964] Lustre: 101487:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 2945.690314] LustreError: 101487:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2945.702488] LustreError: 101487:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2946.023660] LustreError: 101487:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2946.038947] LustreError: 101487:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2953.073807] Lustre: 101487:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2953.084698] Lustre: 101487:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 24 previous similar messages [ 2953.737930] LustreError: 101487:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2953.750577] LustreError: 101487:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2954.079150] LustreError: 101487:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2954.097258] LustreError: 101487:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2969.328790] Lustre: 101487:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2969.351313] Lustre: 101487:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 50 previous similar messages [ 2970.001425] LustreError: 101487:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2970.015862] LustreError: 101487:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2970.335371] LustreError: 101487:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2970.344466] LustreError: 101487:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2992.047967] Lustre: 101487:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2992.058131] Lustre: 101487:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2992.064273] Lustre: 101487:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client d049c5a4-6bf5-44fb-8aee-20639e5ae012@192.168.202.48@tcp [ 2992.077838] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2992.084786] Lustre: 101487:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@000000005483de0a x1875668758498048/t0(8589934767) o101->d049c5a4-6bf5-44fb-8aee-20639e5ae012@192.168.202.48@tcp:435/0 lens 592/0 e 0 to 0 dl 1788778145 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2992.134917] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2992.148552] Lustre: lustre-MDT0000: Denying connection for new client d049c5a4-6bf5-44fb-8aee-20639e5ae012 (at 192.168.202.48@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2992.171880] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2992.208117] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2992.220209] Lustre: Skipped 2 previous similar messages [ 2992.224435] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2992.288370] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:76 to 0x280000401:577) [ 2992.290137] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:45 to 0x2c0000401:545) [ 3007.456412] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3007.467316] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3012.288760] Lustre: server umount lustre-OST0000 complete [ 3014.115248] LustreError: 97087:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3014.136181] LustreError: 97087:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 35 previous similar messages [ 3021.863597] Lustre: server umount lustre-OST0001 complete [ 3032.501345] Lustre: server umount lustre-MDT0000 complete [ 3036.594941] LustreError: 95244:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788778179 with bad export cookie 17393430377734293858 [ 3036.609350] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3036.613541] LustreError: 95244:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3037.178926] Lustre: server umount lustre-MDT0001 complete [ 3047.688719] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 06:49:48 (1788778188) [ 3059.188109] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3059.204767] Lustre: *** cfs_fail_loc=197, val=0*** [ 3075.231745] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3080.351561] LustreError: 103462:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.148@tcp: failed processing log, type 4: rc = -110 [ 3085.489150] Lustre: *** cfs_fail_loc=197, val=0*** [ 3085.490549] Lustre: Skipped 65 previous similar messages [ 3106.083189] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3106.087089] Lustre: Skipped 3 previous similar messages [ 3114.459535] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 3125.391210] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3127.807349] Lustre: server umount lustre-OST0000 complete [ 3139.603608] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 06:51:20 (1788778280) [ 3162.697627] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3163.158435] LustreError: 105089:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3163.178347] LustreError: 105089:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 3167.520667] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 3177.231863] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3181.097050] LustreError: 105120:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3181.103723] LustreError: 105120:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3181.110411] LustreError: 105120:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 18, retries 0, failed: rc = -5 [ 3181.361493] Lustre: server umount lustre-MDT0000 complete [ 3219.492833] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [ 3229.780890] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 3275.957635] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3276.254976] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3276.295898] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3276.405812] Lustre: lustre-MDT0000: new disk, initializing [ 3276.571601] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3283.080242] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 3294.851971] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3295.016122] Lustre: 109294:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3295.356939] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3302.872541] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 3315.926072] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3322.193629] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3331.484147] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3331.748206] Lustre: 111270:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3339.742596] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 3341.351213] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3352.175133] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3359.059766] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3359.349134] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3365.298906] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3365.657705] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3372.009798] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3372.021000] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3372.027724] LustreError: Skipped 5 previous similar messages [ 3372.031091] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3372.031103] Lustre: Skipped 11 previous similar messages [ 3372.071436] Lustre: Skipped 6 previous similar messages [ 3374.351707] Lustre: server umount lustre-OST0000 complete [ 3386.472226] Lustre: server umount lustre-MDT0000 complete [ 3391.242644] LustreError: 109289:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788778534 with bad export cookie 17393430377734304414 [ 3391.253229] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3391.280725] LustreError: 109289:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3391.810352] Lustre: server umount lustre-MDT0001 complete [ 3402.070392] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 06:55:43 (1788778543) [ 3412.379640] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 3416.149853] Key type lgssc unregistered [ 3416.416878] LNet: 113760:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3416.422516] LNetError: 113760:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3416.437222] LNet: Removed LNI 192.168.202.148@tcp [ 3417.387464] Key type .llcrypt unregistered [ 3417.389392] Key type ._llcrypt unregistered [ 3435.798775] Key type ._llcrypt registered [ 3435.801245] Key type .llcrypt registered [ 3435.960463] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [ 3453.091463] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 3454.641563] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3454.735179] alg: No test for adler32 (adler32-zlib) [ 3456.152929] Lustre: Lustre: Build Version: 2.17.57_103_g80273de [ 3456.399362] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 3458.063275] Key type lgssc registered [ 3459.902621] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3507.379757] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3507.432088] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3508.762681] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3508.809863] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3508.927482] Lustre: lustre-MDT0000: new disk, initializing [ 3509.100705] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3509.136928] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3515.679855] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 3529.063112] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3529.337408] Lustre: lustre-OST0000: new disk, initializing [ 3529.354751] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3529.376667] Lustre: Skipped 1 previous similar message [ 3529.399151] Lustre: 117930:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3529.508633] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3538.090945] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 3539.694149] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3539.716786] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3539.789956] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3550.825363] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3551.186461] Lustre: lustre-OST0001: new disk, initializing [ 3551.196401] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3551.204797] Lustre: 118796:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3551.375695] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3558.045242] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3558.048845] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3558.088944] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3558.203633] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 3568.613262] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3568.630464] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3568.637589] Lustre: Skipped 1 previous similar message [ 3568.645880] Lustre: Skipped 1 previous similar message [ 3573.728986] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3573.731806] Lustre: Skipped 1 previous similar message [ 3574.304472] Lustre: server umount lustre-MDT0000 complete [ 3582.468618] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3588.338936] LustreError: 117116:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788778731 with bad export cookie 10002336188129737379 [ 3588.347129] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3588.367199] LustreError: 117116:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3588.487344] Lustre: server umount lustre-OST0000 complete [ 3593.683701] Lustre: server umount lustre-OST0001 complete [ 3605.450093] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 06:59:06 (1788778746) [ 3632.048951] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 06:59:33 (1788778773) [ 3644.196793] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 3668.159189] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 07:00:09 (1788778809) [ 3713.676147] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [ 3730.594577] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 3772.066322] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3772.697751] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3772.785752] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3773.062887] Lustre: lustre-MDT0000: new disk, initializing [ 3773.258201] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3773.318044] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3781.988529] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 3794.498357] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3794.685904] Lustre: 126585:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3794.790854] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3794.799527] Lustre: Skipped 1 previous similar message [ 3794.911393] Lustre: lustre-MDT0001: new disk, initializing [ 3795.051843] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3795.098994] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3795.113598] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3799.691885] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 3811.518885] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3819.159207] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3827.184582] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3827.507878] Lustre: lustre-OST0000: new disk, initializing [ 3827.512541] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3827.518289] Lustre: 128555:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3827.647840] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3833.402025] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3833.413886] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3833.520102] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3834.849540] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 3844.516335] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3849.492888] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3849.724465] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3854.838986] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3855.109215] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3859.428456] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3859.434938] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3859.446533] LustreError: Skipped 1 previous similar message [ 3859.447802] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3859.482576] Lustre: Skipped 1 previous similar message [ 3863.906927] Lustre: server umount lustre-OST0000 complete [ 3869.680067] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3869.695556] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3869.698044] Lustre: Skipped 1 previous similar message [ 3873.849618] Lustre: server umount lustre-MDT0000 complete [ 3877.693198] LustreError: 128554:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788779020 with bad export cookie 10002336188129738604 [ 3877.696115] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3877.713517] LustreError: 128554:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3877.940581] Lustre: server umount lustre-MDT0001 complete [ 3886.550520] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 07:03:48 (1788779028) [ 3894.158270] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3894.755888] LustreError: 130964:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3894.779390] LustreError: 130964:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 3894.852274] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3899.536060] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 3899.894990] LustreError: 130965:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3903.975877] LustreError: 130965:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3909.095350] LustreError: 130964:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3909.532421] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3916.111812] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 3926.846233] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3933.056856] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3940.450683] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3940.847664] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3940.858403] Lustre: Skipped 1 previous similar message [ 3948.574290] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 3956.475214] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3972.295717] Lustre: *** cfs_fail_loc=159, val=0*** [ 3988.665229] Lustre: lustre-MDT0000: Client b5a1647a-4d82-4162-97c0-252d50d63126 (at 192.168.202.48@tcp) reconnecting [ 3990.482606] Lustre: *** cfs_fail_loc=159, val=0*** [ 3990.488784] Lustre: Skipped 5 previous similar messages [ 4006.045658] Lustre: lustre-MDT0000: Client b5a1647a-4d82-4162-97c0-252d50d63126 (at 192.168.202.48@tcp) reconnecting [ 4015.588202] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4015.597960] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4015.624344] Lustre: Skipped 1 previous similar message [ 4015.645228] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4015.670504] Lustre: Skipped 1 previous similar message [ 4019.902283] Lustre: server umount lustre-OST0000 complete [ 4025.824874] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4025.830170] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4025.859244] Lustre: Skipped 2 previous similar messages [ 4025.890635] Lustre: Skipped 1 previous similar message [ 4030.669475] Lustre: server umount lustre-MDT0000 complete [ 4030.948044] LustreError: 133530:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4030.975759] LustreError: 133530:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4034.445206] LustreError: 130946:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788779177 with bad export cookie 10002336188129739486 [ 4034.458831] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4035.138894] Lustre: server umount lustre-MDT0001 complete [ 4046.620867] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 4049.330400] Key type lgssc unregistered [ 4049.590653] LNet: 134762:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4049.599482] LNetError: 134762:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4049.626863] LNet: Removed LNI 192.168.202.148@tcp [ 4050.931684] Key type .llcrypt unregistered [ 4050.939434] Key type ._llcrypt unregistered [ 4074.782427] Key type ._llcrypt registered [ 4074.785940] Key type .llcrypt registered [ 4074.883937] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 4076.129997] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4076.192727] alg: No test for adler32 (adler32-zlib) [ 4077.444935] Lustre: Lustre: Build Version: 2.17.57_103_g80273de [ 4077.734609] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 4079.623950] Key type lgssc registered [ 4082.029667] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4093.274570] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4093.302954] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4095.040924] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4100.178720] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 4105.185530] LustreError: 136173:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4105.205722] LustreError: 136173:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4110.304822] LustreError: 136174:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4110.673316] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4111.034200] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4115.708925] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 4124.115702] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4130.271520] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4139.502634] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4139.938602] LustreError: 137996:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4140.194629] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4145.656854] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4146.252269] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 4158.033449] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4169.521058] Lustre: *** cfs_fail_loc=159, val=0*** [ 4185.732473] Lustre: lustre-MDT0000: Client a0180e9d-cd87-4742-83c6-96347951008e (at 192.168.202.48@tcp) reconnecting [ 4187.866236] Lustre: *** cfs_fail_loc=159, val=0*** [ 4203.189917] Lustre: lustre-MDT0001: Client a0180e9d-cd87-4742-83c6-96347951008e (at 192.168.202.48@tcp) reconnecting [ 4204.876750] Lustre: *** cfs_fail_loc=159, val=0*** [ 4204.882508] Lustre: Skipped 3 previous similar messages [ 4220.125345] Lustre: lustre-MDT0001: Client a0180e9d-cd87-4742-83c6-96347951008e (at 192.168.202.48@tcp) reconnecting [ 4237.166042] Lustre: 139231:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4240.679259] Lustre: *** cfs_fail_loc=159, val=0*** [ 4240.691438] Lustre: Skipped 5 previous similar messages [ 4256.461399] Lustre: lustre-MDT0000: Client c32ef8fb-655c-49fe-82a5-e7927a120a59 (at 192.168.202.48@tcp) reconnecting [ 4258.304601] Lustre: *** cfs_fail_loc=159, val=0*** [ 4258.310275] Lustre: Skipped 13 previous similar messages [ 4274.903595] Lustre: lustre-MDT0000: Client c32ef8fb-655c-49fe-82a5-e7927a120a59 (at 192.168.202.48@tcp) reconnecting [ 4277.138765] Lustre: 139521:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4299.233627] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4299.233916] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4299.246444] LustreError: Skipped 1 previous similar message [ 4299.247514] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4299.283931] Lustre: Skipped 1 previous similar message [ 4301.007540] Lustre: server umount lustre-OST0000 complete [ 4305.380648] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4305.396231] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4305.427919] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4305.438738] Lustre: Skipped 1 previous similar message [ 4309.477951] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4309.483326] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4309.520933] Lustre: Skipped 1 previous similar message [ 4311.557140] Lustre: server umount lustre-MDT0000 complete [ 4314.594123] LustreError: 136168:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4314.627658] LustreError: 136168:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4315.843692] LustreError: 136155:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788779459 with bad export cookie 4051835669462077546 [ 4315.847950] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4315.852713] LustreError: 136155:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4316.684751] Lustre: server umount lustre-MDT0001 complete [ 4330.479264] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 4333.955276] Key type lgssc unregistered [ 4334.252909] LNet: 140601:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4334.258556] LNetError: 140601:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4335.272252] LNet: Removed LNI 192.168.202.148@tcp [ 4336.288438] Key type .llcrypt unregistered [ 4336.290635] Key type ._llcrypt unregistered [ 4362.078337] Key type ._llcrypt registered [ 4362.082177] Key type .llcrypt registered [ 4362.271659] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 4363.317931] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4363.349383] alg: No test for adler32 (adler32-zlib) [ 4364.498869] Lustre: Lustre: Build Version: 2.17.57_103_g80273de [ 4364.707445] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 4366.375171] Key type lgssc registered [ 4367.566139] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4376.682861] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4376.755208] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4378.211309] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4382.817447] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 4388.842637] LustreError: 142012:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4388.883879] LustreError: 142012:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4393.139563] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4393.763279] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4399.919446] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 4409.588155] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4417.935601] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4425.119421] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4425.319341] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4430.977762] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 4438.666794] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4439.404264] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4448.946581] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4455.403044] Lustre: 144614:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4458.427300] Lustre: 144662:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4460.080709] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4461.106562] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4464.379205] Lustre: 144808:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4465.286554] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4466.347674] Lustre: 144916:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4474.894875] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4474.900480] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4474.902599] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4474.907955] Lustre: Skipped 1 previous similar message [ 4476.710218] Lustre: server umount lustre-OST0000 complete [ 4480.993758] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4481.005681] LustreError: Skipped 1 previous similar message [ 4481.010051] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4481.043587] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4481.059116] Lustre: Skipped 1 previous similar message [ 4485.109234] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4485.116629] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4486.991241] Lustre: server umount lustre-MDT0000 complete [ 4490.210176] LustreError: 142006:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4490.232659] LustreError: 142006:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4491.219509] LustreError: 141993:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788779634 with bad export cookie 12657051187986539631 [ 4491.222116] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4491.231453] LustreError: 141993:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4491.758792] Lustre: server umount lustre-MDT0001 complete [ 4503.622469] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 4506.840995] Key type lgssc unregistered [ 4507.224799] LNet: 145967:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4507.229031] LNetError: 145967:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4507.249955] LNet: Removed LNI 192.168.202.148@tcp [ 4508.136609] Key type .llcrypt unregistered [ 4508.142240] Key type ._llcrypt unregistered [ 4533.992134] Key type ._llcrypt registered [ 4533.993479] Key type .llcrypt registered [ 4534.076158] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 4534.991770] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4535.109715] alg: No test for adler32 (adler32-zlib) [ 4536.117306] Lustre: Lustre: Build Version: 2.17.57_103_g80273de [ 4536.364653] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 4538.000040] Key type lgssc registered [ 4539.390390] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4548.528742] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4548.573676] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4550.267259] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4555.332617] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 4559.843214] LustreError: 147379:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4559.856351] LustreError: 147379:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4563.933539] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4564.371636] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4569.413458] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 4577.193134] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4583.459874] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4590.383791] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4590.801575] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4594.930508] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4597.807317] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 4606.272698] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4609.274595] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4616.004110] Lustre: *** cfs_fail_loc=159, val=0*** [ 4632.312467] Lustre: lustre-MDT0000: Client d8fa9b7b-7f88-4ada-81d8-f32d9e84cb16 (at 192.168.202.48@tcp) reconnecting [ 4650.466255] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4650.505796] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4650.525752] Lustre: Skipped 1 previous similar message [ 4655.587254] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4655.952604] Lustre: server umount lustre-OST0000 complete [ 4661.731986] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4661.751181] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4661.782315] Lustre: Skipped 1 previous similar message [ 4661.791262] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4661.797108] Lustre: Skipped 1 previous similar message [ 4665.838742] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4665.842556] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4667.939070] Lustre: server umount lustre-MDT0000 complete [ 4670.953797] LustreError: 147379:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4670.989658] LustreError: 147379:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4671.878449] LustreError: 147359:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788779815 with bad export cookie 15830330761889613147 [ 4671.893416] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4671.903120] LustreError: 147359:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4672.426725] Lustre: server umount lustre-MDT0001 complete [ 4684.339475] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 4687.917575] Key type lgssc unregistered [ 4688.325401] LNet: 151045:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4688.333270] LNetError: 151045:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4688.343985] LNet: Removed LNI 192.168.202.148@tcp [ 4689.337688] Key type .llcrypt unregistered [ 4689.342607] Key type ._llcrypt unregistered [ 4714.513502] Key type ._llcrypt registered [ 4714.518193] Key type .llcrypt registered [ 4714.662107] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 4715.778445] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4716.056210] alg: No test for adler32 (adler32-zlib) [ 4717.152377] Lustre: Lustre: Build Version: 2.17.57_103_g80273de [ 4717.424593] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 4719.103457] Key type lgssc registered [ 4720.425389] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4728.920996] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4728.961647] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4730.510173] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4735.606910] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 4740.065946] LustreError: 152455:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4740.080921] LustreError: 152455:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4744.324713] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4744.668465] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4749.383811] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 4756.774361] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4763.446540] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4770.738488] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4771.151373] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4775.288310] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4777.798650] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 4785.286793] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4789.627141] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4811.102369] Lustre: lustre-MDT0001: evicting 0c08cd84-90c9-4e84-bd49-6122eb1bde60 (at 192.168.202.48@tcp) by administrative request [ 4811.115800] Lustre: Skipped 1 previous similar message [ 4840.930895] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4840.945256] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4840.945303] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4840.946975] Lustre: Skipped 1 previous similar message [ 4846.053326] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4846.056949] Lustre: Skipped 1 previous similar message [ 4846.778518] Lustre: server umount lustre-OST0000 complete [ 4851.176893] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4851.183212] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4851.188217] Lustre: Skipped 1 previous similar message [ 4851.197749] Lustre: Skipped 2 previous similar messages [ 4856.289564] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4856.292909] Lustre: Skipped 1 previous similar message [ 4856.971361] Lustre: server umount lustre-MDT0000 complete [ 4860.917965] LustreError: 152437:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788780004 with bad export cookie 4356865492064715266 [ 4860.928111] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4860.933566] LustreError: 152437:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4861.314692] Lustre: server umount lustre-MDT0001 complete [ 4875.500982] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 4880.303388] Key type lgssc unregistered [ 4880.777613] LNet: 156524:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4880.794223] LNetError: 156524:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4880.820372] LNet: Removed LNI 192.168.202.148@tcp [ 4881.963883] Key type .llcrypt unregistered [ 4881.966217] Key type ._llcrypt unregistered [ 4958.870967] Key type ._llcrypt registered [ 4958.872746] Key type .llcrypt registered [ 4959.024271] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 4960.741923] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4960.782432] alg: No test for adler32 (adler32-zlib) [ 4961.895440] Lustre: Lustre: Build Version: 2.17.57_103_g80273de [ 4962.288081] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 4964.025157] Key type lgssc registered [ 4965.237233] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4972.872249] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4972.890028] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4974.372906] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4978.627280] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 4984.802558] LustreError: 159010:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4984.828338] LustreError: 159010:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4986.770179] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4987.177723] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4991.997172] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 4999.200702] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5004.750977] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5011.333138] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5011.723914] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5012.797096] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5018.118180] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 5018.924940] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 5026.688678] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5038.568033] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5038.575228] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5038.588582] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5039.595790] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5039.598323] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5039.603248] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5044.716279] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5044.726852] Lustre: Skipped 2 previous similar messages [ 5049.826086] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5049.839922] Lustre: Skipped 1 previous similar message [ 5050.336996] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5050.541884] Lustre: server umount lustre-OST0000 complete [ 5054.945262] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5054.958130] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5054.966759] Lustre: Skipped 1 previous similar message [ 5060.396673] Lustre: server umount lustre-MDT0000 complete [ 5064.529669] LustreError: 158991:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788780207 with bad export cookie 18342837304078865707 [ 5064.543882] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5064.554166] LustreError: 158991:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5064.954572] Lustre: server umount lustre-MDT0001 complete [ 5076.669499] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 5079.530170] Key type lgssc unregistered [ 5079.873846] LNet: 162554:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5079.882963] LNetError: 162554:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5079.919799] LNet: Removed LNI 192.168.202.148@tcp [ 5080.865201] Key type .llcrypt unregistered [ 5080.866473] Key type ._llcrypt unregistered [ 5106.506913] Key type ._llcrypt registered [ 5106.522594] Key type .llcrypt registered [ 5106.812639] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 5109.264725] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5109.357869] alg: No test for adler32 (adler32-zlib) [ 5110.834661] Lustre: Lustre: Build Version: 2.17.57_103_g80273de [ 5111.474488] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 5113.258679] Key type lgssc registered [ 5115.355457] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5130.654367] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 07:24:32 (1788780272) [ 5148.356178] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [ 5159.655665] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 5196.775126] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5196.787361] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5198.011276] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5198.093047] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5198.212314] Lustre: lustre-MDT0000: new disk, initializing [ 5198.278980] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5198.319643] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5203.917263] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 5215.817609] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5215.993203] Lustre: 166817:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5216.080932] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5216.085773] Lustre: Skipped 1 previous similar message [ 5216.156754] Lustre: lustre-MDT0001: new disk, initializing [ 5216.236098] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5216.272441] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5216.282110] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5222.059854] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 5232.971705] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5238.375366] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5246.315578] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5246.600694] Lustre: lustre-OST0000: new disk, initializing [ 5246.604535] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5246.612944] Lustre: 168787:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5246.709710] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5252.530858] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 5254.792272] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5254.804858] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5254.931127] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5262.713484] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5275.621814] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5275.624507] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5275.628850] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5275.637120] LustreError: Skipped 1 previous similar message [ 5275.678721] Lustre: Skipped 1 previous similar message [ 5278.383698] Lustre: server umount lustre-OST0000 complete [ 5282.791914] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5282.803953] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5282.826909] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5282.836295] Lustre: Skipped 1 previous similar message [ 5285.856348] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5285.858590] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5288.908081] Lustre: server umount lustre-MDT0000 complete [ 5292.797099] LustreError: 166810:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788780435 with bad export cookie 198894044896533573 [ 5292.807208] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5292.817203] LustreError: 166810:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5293.086782] Lustre: server umount lustre-MDT0001 complete [ 5304.626912] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 5308.077808] Key type lgssc unregistered [ 5308.575661] LNet: 170690:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5308.589924] LNetError: 170690:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5308.614406] LNet: Removed LNI 192.168.202.148@tcp [ 5309.707496] Key type .llcrypt unregistered [ 5309.709037] Key type ._llcrypt unregistered [ 5337.750867] Key type ._llcrypt registered [ 5337.752877] Key type .llcrypt registered [ 5337.845777] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 5339.030664] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5339.239268] alg: No test for adler32 (adler32-zlib) [ 5340.392854] Lustre: Lustre: Build Version: 2.17.57_103_g80273de [ 5340.747558] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 5342.423347] Key type lgssc registered [ 5343.719794] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5350.536138] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5350.550826] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5351.920608] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5356.584764] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 5362.144734] LustreError: 172104:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5362.168805] LustreError: 172104:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5365.590785] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5366.027774] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5371.892794] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 5379.985710] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5387.241307] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5393.310639] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5393.738428] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5397.824307] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5400.074891] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 5407.067658] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5416.390552] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5416.413888] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5416.437207] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5416.448930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.463886] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5417.515225] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5417.532311] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5417.545055] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5417.556151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.567317] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5418.539874] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5418.556979] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5418.566316] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5418.569376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.578514] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5419.440626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.367599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.096274] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5421.100885] Lustre: Skipped 2 previous similar messages [ 5421.111895] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5421.120522] Lustre: Skipped 2 previous similar messages [ 5421.123146] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5421.126830] Lustre: Skipped 2 previous similar messages [ 5421.130243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.140752] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5421.143036] Lustre: Skipped 2 previous similar messages [ 5422.071807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.900504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.802208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.826032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.050823] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5426.058471] Lustre: Skipped 4 previous similar messages [ 5426.088484] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5426.105279] Lustre: Skipped 4 previous similar messages [ 5426.113269] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5426.116282] Lustre: Skipped 4 previous similar messages [ 5426.127486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.148408] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5426.155850] Lustre: Skipped 4 previous similar messages [ 5427.022577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.053502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.962032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.058502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.078037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.151565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.057931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.061133] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5434.069016] Lustre: Skipped 7 previous similar messages [ 5434.087168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.785836] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5434.794320] Lustre: Skipped 8 previous similar messages [ 5434.798547] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5434.801985] Lustre: Skipped 8 previous similar messages [ 5434.804674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.811617] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5434.814105] Lustre: Skipped 8 previous similar messages [ 5435.842827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.827329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.774550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.588556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.531358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.431562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.390648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.182741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.963890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.850123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.680341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.419250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.147385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.920612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.844834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.515382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.334460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.172528] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5450.177134] Lustre: Skipped 18 previous similar messages [ 5450.189813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.031699] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5451.039823] Lustre: Skipped 18 previous similar messages [ 5451.043632] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5451.046324] Lustre: Skipped 18 previous similar messages [ 5451.049733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.062288] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5451.074929] Lustre: Skipped 18 previous similar messages [ 5452.070369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.940190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.740519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.404126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.259463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.230419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.003201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.853269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.755951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.777444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.527587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.552860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.528664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.366852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.118207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.908985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.658713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.376386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.180362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.971696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.884411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.816242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.665323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.512736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.372443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.361741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.089677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.074105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.028489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.957489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.922603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.857569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.615485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.463243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.474301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.222835] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5482.232740] Lustre: Skipped 36 previous similar messages [ 5482.248600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.243449] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5483.251829] Lustre: Skipped 36 previous similar messages [ 5483.254618] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5483.257738] Lustre: Skipped 36 previous similar messages [ 5483.262272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.272798] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5483.278165] Lustre: Skipped 36 previous similar messages [ 5484.134731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.011769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.724564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.536598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.334730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.188341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.008797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.841442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.595649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.439495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.196517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.063974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.969837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.949287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.971267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.113940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.054606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.187845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.294339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.207391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.147067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.294727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.300398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.253824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.055662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.925479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.881248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.004652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.012580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.030705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.002283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.116540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.111984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.041176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.976431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.928318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.705269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.589308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.521418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.564385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.482613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.529318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.459625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.473191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.294172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.176713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.006641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.819878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.730058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.651732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.548127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.434042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.321316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.219083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.249731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.179027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.093415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.057985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.912684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.116251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.013909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.001511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.096502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.000171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.929412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.692863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.602772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.530956] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5546.534809] Lustre: Skipped 68 previous similar messages [ 5546.576663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.510073] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5547.517692] Lustre: Skipped 68 previous similar messages [ 5547.526426] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5547.529981] Lustre: Skipped 68 previous similar messages [ 5547.533180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.543102] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5547.547167] Lustre: Skipped 68 previous similar messages [ 5548.365434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.405486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.278919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.295320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.222671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.063241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.028391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.986455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.144156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.201428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.108242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.097728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.044197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.106824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.054805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.146742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.190495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.327219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.443654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.407535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.312493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.280970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.346700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.218989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.230266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.340598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.307710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.205263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.375438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.183826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.071847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.175689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.013962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.126853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.061828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.032805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.915704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.928868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.673257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.584114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.465903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.398462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.215808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.340462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.257952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.336412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.426751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.397057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.385656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.154274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.059202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.149974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.090817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.035449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.938895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.986908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.770975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.732870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.538476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.519787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.586712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.558991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.432415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.296035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.241564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.431214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.538300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.697996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.574662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.434402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.229903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.097336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.005789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.020834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.899510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.901428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.907479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.937565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.845570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.790403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.599223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.494757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.190818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.032030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.890049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.827396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.704557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.639387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.586957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.528973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.446860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.527598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.472979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.366747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.156953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.113647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.104768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.213145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.153467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.249177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.883996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.773437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.580452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.399208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.207661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.285618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.121742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.979132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.878643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.908710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.681543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.367676] LustreError: 172123:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5653.374447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.377893] LustreError: 172123:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5654.174978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.866181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.768041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.515400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.301843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.057809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.996162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.995508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.990632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.017251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.877591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.819292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.643931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.388349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.369147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.068794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.995172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.790134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.578659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.332225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.155625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.048424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.011809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.880979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.774569] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5674.780354] Lustre: Skipped 137 previous similar messages [ 5674.797575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.691590] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5675.698713] Lustre: Skipped 137 previous similar messages [ 5675.702534] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5675.707857] Lustre: Skipped 137 previous similar messages [ 5675.711772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.720034] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5675.722941] Lustre: Skipped 137 previous similar messages [ 5676.510333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.438513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.453787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.524853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.431432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.418343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.290490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.404576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.316940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.386570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.247557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.219377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.923818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.934105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.924396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.827638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.774696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.761771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.635853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.810409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.658650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.487028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.396520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.422582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.173559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.122086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.065637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.011931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.801193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.690646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.675567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.566431] LustreError: 172123:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5705.577015] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.579133] LustreError: 172123:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5706.491430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.451344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.360500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.143176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.901271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.791632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.696525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.573883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.423932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.315242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.109663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.013163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.942601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.868656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.601814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.735551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.490659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.190066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.136649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.012313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.029311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.935597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.777827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.714949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.602434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.348680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.224031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.255460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.141467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.942958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.848716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.608740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.417971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.220897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.151388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.085570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.003790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.789522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.549950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.444578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.449916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.348082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.203183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.070562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.084736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.139586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.034343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.043485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.838867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.884356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.017777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.977038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.085312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.974631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.161148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.980761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.989612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.753966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.719990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.587109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.570984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.516311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.349639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.337323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.153581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.987709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.715692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.623942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.320274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.316483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.078865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.302652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.200376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.114687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.782953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.817547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.759321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.741858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.824853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.881575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.835570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.730841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.608211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.611733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.427644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.380818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.096194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.139826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.864982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.723844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.747079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.606875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.452844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.129039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.881656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.677078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.477977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.351434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.430308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.278037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.164641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.368982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.274437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.014493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.856355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.818887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.531201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.331802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.246490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.958640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.793335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.561310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.288766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.274271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.063704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.105404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.985469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.908568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.877265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.933063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.808323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.595746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.516548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.325066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.249265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.309033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.308677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.075651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.005583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.720267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.553910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.456486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.448796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.325482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.419450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.429678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.297984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.058710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.921200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.774919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.724429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.656422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.703936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.718526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.871834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.080001] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.112050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.212714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.362233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.346718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.233969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.082435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.059847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.860667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.763781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.765592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.660871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.520935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.264271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.913302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.568731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.214255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.948481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.588458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.416877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.240245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.103206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.025426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.884647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.606433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.509721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.336980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.274834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.255065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.254158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.273446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.459769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.588356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.669747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.483139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.386530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.330821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.178918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.245692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.094504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.972285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.906677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.132574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.919208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.803207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.692637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.434731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.097943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.738562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.477860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.229935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.886460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.632097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.320054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.987234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.643033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.347734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.942388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.685859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.300028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.045552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.714737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.478133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.261096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.130307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.856289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.693628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.422230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.360187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.316321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.383111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.307756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.186268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.986809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.830322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.897489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.866448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.662691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.628616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.508419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.540193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.599548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.526072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.464450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.346440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.401859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.350314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.290517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.338208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.254753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.255241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.136524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.182773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.146526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.102285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.109857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.043608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.940597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.900872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.902068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.701454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.569228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.507825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.485402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.402443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.326150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.134117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.920880] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5930.925552] Lustre: Skipped 285 previous similar messages [ 5930.942297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.615681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.283359] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5932.296070] Lustre: Skipped 286 previous similar messages [ 5932.299775] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5932.305970] Lustre: Skipped 286 previous similar messages [ 5932.309279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.320978] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5932.323460] Lustre: Skipped 286 previous similar messages [ 5933.068573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.893100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.718337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.508941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.222524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.912995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.692777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.529860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.296544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.088917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.817869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.602842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.435835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.225650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.982738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.771138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.445967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.302205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.091197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.137346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.931555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.913650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.759667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.430328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.294611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.085521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.887602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.613853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.580310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.552488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.832125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.688087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.941744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.947503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.806324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.619574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.585757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.580256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.470183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.870792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.785376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.784088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.595232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.569885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.476932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.394888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.365526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.384553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.397619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.370672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.320942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.040786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.094981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.974118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.764273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.697133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.626057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.340868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.008709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.840111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.661317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.553586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.255617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.932921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.742948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.316442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.145103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.840393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.645436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.477943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.208289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.121587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.850040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.712643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.379969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.199754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.037910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.851454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.573376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.344681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.325584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.215475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.097232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.078771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.948513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.798516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.788232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.701782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.436055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.188656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.000673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.849053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.685156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.532223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.312594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.108684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.098780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.011204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.797548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.668822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.440550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.226310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.030917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.003781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.915482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.838367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.742331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.490557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.170614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.979857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.764956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.488843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.165784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.938903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.641559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.581556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.607669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.530391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.401733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.232924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.164525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.003732] LustreError: 172123:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6036.017969] LustreError: 172123:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6036.023836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.807478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.720219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.533230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.761490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.644117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.634297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.600597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.521527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.327551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.233215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.219243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.070982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.906989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.937484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.771229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.780741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.713152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.559496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.491330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.400453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.220778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.025807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.952800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.831200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.735346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.637513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.548765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.452288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.264888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.338296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.096070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.960530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.724300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.464978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.216515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.942810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.635816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.346335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.092324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.875966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.602858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.389911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.260605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.157353] LustreError: 172123:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6074.175702] LustreError: 172123:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6074.178369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.936881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.852878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.680213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.593069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.446972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.177131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.000661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.842908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.680664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.406265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.278277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.124880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.915392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.902964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.717450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.436560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.184646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.081568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.004592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.881897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.819765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.796762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.691353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.660793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.594236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.736197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.648195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.578674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.469464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.146762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.972821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.694909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.489879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.277681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.212604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.185835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.044860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.000078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.835353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.656454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.315809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.268068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.019554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.928940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.726892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.600855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.495337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.399737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.207190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.983265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.867115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.729490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.635280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.526750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.376768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.144487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.942781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.782702] LustreError: 172123:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6123.797514] LustreError: 172123:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6123.799274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.691497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.595648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.342164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.119848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.992835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.810738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.794916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.685363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.466257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.424426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.151856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.094459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.991652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.977608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.750564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.721428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.515411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.440940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.332892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.300099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.067608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.098268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.107671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.873023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.740343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.642017] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.346874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.203272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.977173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.768792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.576637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.334086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.195705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.122240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.973108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.715240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.602227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.565844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.357049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.268865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.186036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.135739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.881580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.674825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.585919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.491797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.408290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.253560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.111548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.865877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.748516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.649624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.559796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.648346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.527443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.512471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.559653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.547418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.456699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.328928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.134071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.025145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.814357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.893113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.866076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.921074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.978988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.005795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.031608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.977289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.021748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.775057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.472913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.377817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.066921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.057544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.002934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.721213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.503679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.464118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.438893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.384535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.369893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.544936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.418778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.406614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.755902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.005547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.785997] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.942214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.818902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.912494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.940094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.678160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.397752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.079559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.994682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.579649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.168707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.870071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.535359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.443230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.448126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.311571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.346639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.166227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.872964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.730784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.846621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.613538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.641903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.547016] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.367262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.272360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.194947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.167037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.306327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.073679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.878633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.676883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.428325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.225862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.120854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.948716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.877279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.840708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.834899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.805129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.813380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.706958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.644611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.418175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.442974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.211939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.033754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.088930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.041286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.983080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.883234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.912205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.762823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.891302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.693900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.544407] LustreError: 172123:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6252.552107] LustreError: 172123:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6252.555561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.377148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.223913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.196481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.162250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.111820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.006138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.896469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.850371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.589864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.431074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.246441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.891176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.699329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.322887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.049436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.751119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.432988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.210220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.907208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.777433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.696849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.407312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.233523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.973899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.690626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.405061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.089398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.827860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.428841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.208961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.834399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.643991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.417386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.197814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.104281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.874209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.735411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.697773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.499085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.264024] LustreError: 172123:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6284.275783] LustreError: 172123:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6284.287176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.189250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.279433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.118249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.944799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.730209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.657496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.663848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.545432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.631713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.536720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.430469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.173493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.008573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.777081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.635969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.660794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.605329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.438764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.471043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.261289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.342429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.436847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6620.493314] LustreError: 173926:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6620.509682] LustreError: 172123:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7023.080878] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 7023.081162] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7023.086464] LustreError: Skipped 2 previous similar messages [ 7023.096489] Lustre: Skipped 430 previous similar messages [ 7023.099897] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7024.096244] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7024.100744] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7024.103803] LustreError: Skipped 5 previous similar messages [ 7024.121476] Lustre: Skipped 1 previous similar message [ 7025.834208] Lustre: server umount lustre-OST0000 complete [ 7030.253130] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7030.264733] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7034.343366] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7034.355945] Lustre: Skipped 1 previous similar message [ 7036.244770] Lustre: server umount lustre-MDT0000 complete [ 7039.471860] LustreError: 173052:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7039.516075] LustreError: 173052:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7040.456776] LustreError: 172085:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788782183 with bad export cookie 13939376052294499441 [ 7040.457841] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7040.468490] LustreError: 172085:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7040.811454] Lustre: server umount lustre-MDT0001 complete [ 7054.150230] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 7056.410976] Key type lgssc unregistered [ 7056.650913] LNet: 227292:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7056.668993] LNetError: 227292:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7057.703374] LNet: Removed LNI 192.168.202.148@tcp [ 7058.577762] Key type .llcrypt unregistered [ 7058.580383] Key type ._llcrypt unregistered [ 7087.976538] Key type ._llcrypt registered [ 7087.978544] Key type .llcrypt registered [ 7088.113268] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 7089.306163] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7089.514731] alg: No test for adler32 (adler32-zlib) [ 7090.527703] Lustre: Lustre: Build Version: 2.17.57_103_g80273de [ 7090.757704] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 7092.416802] Key type lgssc registered [ 7093.554706] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7102.539776] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7102.582935] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7103.991542] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7108.765881] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7114.208606] LustreError: 228707:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7114.230696] LustreError: 228707:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7118.662394] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7119.283663] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7125.126278] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7135.736718] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7142.580096] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7150.381678] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7150.761891] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7157.029770] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7164.030942] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7174.439256] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7179.617375] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7182.332625] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7182.351592] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7182.376091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7182.410854] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7182.415382] LustreError: 228726:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7185.557717] Lustre: lustre-OST0000: Not available for connect from 192.168.202.48@tcp (stopping) [ 7185.900612] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7185.910304] Lustre: Skipped 1 previous similar message [ 7190.649759] Lustre: lustre-OST0000: Not available for connect from 192.168.202.48@tcp (stopping) [ 7190.661391] Lustre: Skipped 2 previous similar messages [ 7191.459383] Lustre: server umount lustre-OST0000 complete [ 7202.463141] LustreError: 228726:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7226.255471] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7226.576320] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7233.075541] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7240.942630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7240.966924] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7240.969037] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7240.999194] Lustre: Skipped 1 previous similar message [ 7242.119151] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7244.162794] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7250.880412] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff903dd1b4c000.ost_server_uuid 50 [ 7252.970896] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff903dd1b4c000.ost_server_uuid in FULL state after 0 sec [ 7258.594088] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7258.599968] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7258.621489] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7258.624085] Lustre: Skipped 2 previous similar messages [ 7260.642804] Lustre: server umount lustre-OST0000 complete [ 7266.783966] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7266.785486] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7266.803620] Lustre: Skipped 1 previous similar message [ 7266.831376] Lustre: Skipped 1 previous similar message [ 7271.648410] Lustre: server umount lustre-MDT0000 complete [ 7271.906824] LustreError: 228703:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7271.937061] LustreError: 228703:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7275.714637] LustreError: 228688:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788782418 with bad export cookie 18207575401063932943 [ 7275.717630] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7275.728513] LustreError: 228688:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7285.559679] Lustre: server umount lustre-MDT0001 complete [ 7297.328874] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 7299.964891] Key type lgssc unregistered [ 7300.269441] LNet: 233939:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7300.279679] LNetError: 233939:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7300.296610] LNet: Removed LNI 192.168.202.148@tcp [ 7301.128165] Key type .llcrypt unregistered [ 7301.129984] Key type ._llcrypt unregistered [ 7338.516481] Key type ._llcrypt registered [ 7338.519825] Key type .llcrypt registered [ 7338.628860] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 7339.802703] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7339.990487] alg: No test for adler32 (adler32-zlib) [ 7341.042434] Lustre: Lustre: Build Version: 2.17.57_103_g80273de [ 7341.301025] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 7342.967202] Key type lgssc registered [ 7344.423831] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7353.293979] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7353.314907] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7354.651638] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7358.943549] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7364.069220] LustreError: 235674:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7364.092506] LustreError: 235674:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7368.924639] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7369.199406] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 7369.302336] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7374.842117] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7378.722156] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7385.692725] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7386.027411] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7392.010752] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7399.356210] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7400.915029] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7401.093901] Lustre: lustre-OST0001: new disk, initializing [ 7401.101561] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7401.106512] Lustre: 237991:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7401.195764] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7407.769548] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7407.780826] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7407.943140] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7408.033267] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7419.396535] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7427.611478] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7443.948902] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7443.963587] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7443.980102] Lustre: Skipped 2 previous similar messages [ 7443.993071] Lustre: Skipped 3 previous similar messages [ 7447.850506] Lustre: server umount lustre-MDT0000 complete [ 7449.062354] LustreError: 235669:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7449.080293] LustreError: 235669:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 7451.988710] LustreError: 235654:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788782595 with bad export cookie 4786758374476348286 [ 7451.992849] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7452.002069] LustreError: 235654:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7452.444828] Lustre: server umount lustre-MDT0001 complete [ 7467.367451] Lustre: server umount lustre-OST0000 complete [ 7470.242658] Lustre: 234937:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788782597/real 1788782597] req@ffff935682949c00 x1875674382042624/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788782613 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7470.301101] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7470.328188] Lustre: Skipped 1 previous similar message [ 7471.195283] Lustre: server umount lustre-OST0001 complete [ 7495.284394] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 7506.977324] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7507.578907] LustreError: 241097:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7507.661060] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7512.839301] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7513.063989] LustreError: 241098:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7517.154751] LustreError: 241097:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7522.246943] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7522.712822] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7528.417821] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7531.441801] Lustre: 242241:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7537.567860] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7544.852819] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7547.116115] LustreError: 242595:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7547.130458] LustreError: 242595:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7552.252067] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49760) [ 7554.086284] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7554.475848] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7554.479876] Lustre: Skipped 1 previous similar message [ 7561.436965] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7569.313633] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7572.909781] Lustre: 244109:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7584.226207] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7584.236241] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7584.252221] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7589.579882] Lustre: server umount mylustre-MDT0000 complete [ 7590.376726] LustreError: 244416:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7590.398150] LustreError: 244416:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 7593.327538] LustreError: 241072:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788782736 with bad export cookie 4786758374476351093 [ 7593.329063] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7593.335070] LustreError: 241072:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7593.893271] Lustre: server umount mylustre-MDT0001 complete [ 7608.782938] Lustre: server umount mylustre-OST0000 complete [ 7610.847769] Lustre: 234939:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788782738/real 1788782738] req@ffff9357b3160a80 x1875674382102272/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788782754 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7610.906854] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7610.926758] Lustre: Skipped 3 previous similar messages [ 7612.999680] Lustre: server umount mylustre-OST0001 complete [ 7633.372773] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 7644.800287] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7645.520253] LustreError: 246416:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7645.756279] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7651.966184] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7662.427716] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7668.165684] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7671.182113] Lustre: 247558:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7677.688592] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7683.853105] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7690.491448] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49792) [ 7692.870820] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7698.455507] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 7700.128716] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7709.121062] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7712.900706] Lustre: 249432:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7722.868815] Lustre: server umount tfs-MDT0000 complete [ 7724.006520] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7724.022042] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7724.043061] LustreError: 246413:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7724.044567] Lustre: Skipped 3 previous similar messages [ 7724.091363] LustreError: 246413:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [ 7727.306469] LustreError: 246392:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788782870 with bad export cookie 4786758374476352759 [ 7727.329400] LustreError: 246392:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7727.357370] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7727.902931] Lustre: server umount tfs-MDT0001 complete [ 7742.431311] Lustre: server umount tfs-OST0000 complete [ 7745.503126] Lustre: 234936:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788782872/real 1788782872] req@ffff9357b478b480 x1875674382153344/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788782888 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7745.522718] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7746.530417] Lustre: server umount tfs-OST0001 complete [ 7768.050613] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 7782.157820] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7782.716624] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7782.719489] Lustre: Skipped 3 previous similar messages [ 7787.126322] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7795.700709] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7801.385541] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7804.969970] Lustre: 252821:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7812.368758] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7819.860726] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7827.453626] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49824) [ 7828.712740] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7834.102365] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 7834.488789] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7841.437519] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7845.410766] Lustre: 254692:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7853.878108] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 08:09:55 (1788782995) [ 7872.995874] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7873.003473] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7873.026306] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7873.031821] Lustre: Skipped 4 previous similar messages [ 7879.050182] Lustre: server umount lustre-MDT0000 complete [ 7880.169372] LustreError: 251680:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7880.185401] LustreError: 251680:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 7882.666677] LustreError: 251655:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788783025 with bad export cookie 4786758374476354425 [ 7882.673411] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7882.674067] LustreError: 251655:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 7885.292602] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7885.301765] Lustre: Skipped 5 previous similar messages [ 7889.128613] Lustre: server umount lustre-MDT0001 complete [ 7907.295355] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7907.404958] Lustre: server umount lustre-OST0000 complete [ 7925.215220] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7925.553936] Lustre: server umount lustre-OST0001 complete [ 7932.736829] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [ 7942.381310] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 7979.486458] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7980.170105] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7980.219856] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7980.441364] Lustre: lustre-MDT0000: new disk, initializing [ 7980.616593] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 7980.625885] Lustre: Skipped 3 previous similar messages [ 7980.654278] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7986.198858] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7997.744155] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7997.887776] Lustre: 257878:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7997.963100] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7997.973221] Lustre: Skipped 1 previous similar message [ 7998.057983] Lustre: lustre-MDT0001: new disk, initializing [ 7998.185696] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7998.207099] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8002.522122] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 8014.858132] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8015.169616] Lustre: lustre-OST0000: new disk, initializing [ 8015.172384] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8015.180307] Lustre: 259461:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8017.136085] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8017.153826] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8017.329046] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8023.136482] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 8034.838591] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8035.102279] Lustre: lustre-OST0001: new disk, initializing [ 8035.108529] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8035.118909] Lustre: 260335:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8037.180981] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8037.202740] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8037.345193] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8040.898933] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 8054.491345] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 08:13:16 (1788783196) [ 8070.368596] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 08:13:31 (1788783211) [ 8088.555878] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8088.563640] Lustre: Skipped 6 previous similar messages [ 8088.565266] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8088.571614] Lustre: Skipped 3 previous similar messages [ 8099.296681] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8099.576815] Lustre: server umount lustre-OST0000 complete [ 8109.032465] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8109.048506] Lustre: Skipped 9 previous similar messages [ 8109.589132] Lustre: server umount lustre-MDT0000 complete [ 8112.952269] LustreError: 257871:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788783256 with bad export cookie 4786758374476355979 [ 8112.955030] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8112.967664] LustreError: 257871:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8119.714579] Lustre: server umount lustre-MDT0001 complete [ 8132.061400] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 8207.593642] Lustre: server umount lustre-OST0001 complete [ 8216.594888] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [ 8238.344735] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 8280.631865] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8280.888653] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8280.922601] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8281.009695] Lustre: lustre-MDT0000: new disk, initializing [ 8281.079920] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8281.082857] Lustre: Skipped 3 previous similar messages [ 8281.104846] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8285.728988] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 8296.332427] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8296.455504] Lustre: 265187:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8296.656688] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8301.856489] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 8312.321087] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8318.173938] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8325.862406] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8326.117889] Lustre: lustre-OST0000: new disk, initializing [ 8326.121433] Lustre: Skipped 1 previous similar message [ 8326.125771] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8326.129779] Lustre: Skipped 2 previous similar messages [ 8326.134673] Lustre: 267157:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8327.441625] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8327.448531] Lustre: Skipped 1 previous similar message [ 8327.466521] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8327.567148] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8333.555636] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 8344.064438] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8353.249262] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8353.250344] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8353.265967] LustreError: Skipped 1 previous similar message [ 8353.271636] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8353.271646] Lustre: Skipped 2 previous similar messages [ 8353.311543] Lustre: Skipped 5 previous similar messages [ 8358.380896] LustreError: 267139:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8358.402732] LustreError: 267139:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 11 previous similar messages [ 8358.618308] Lustre: server umount lustre-OST0000 complete [ 8363.494182] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8369.153840] Lustre: server umount lustre-MDT0000 complete [ 8373.366318] LustreError: 265180:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788783516 with bad export cookie 4786758374476357449 [ 8373.373091] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8373.374646] LustreError: 265180:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8373.970663] Lustre: server umount lustre-MDT0001 complete [ 8386.893437] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 8390.462644] Key type lgssc unregistered [ 8390.866957] LNet: 268916:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8390.872653] LNetError: 268916:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8390.908935] LNet: Removed LNI 192.168.202.148@tcp [ 8391.968485] Key type .llcrypt unregistered [ 8391.972727] Key type ._llcrypt unregistered [ 8421.126295] Key type ._llcrypt registered [ 8421.130815] Key type .llcrypt registered [ 8421.285547] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 8422.426699] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8422.626338] alg: No test for adler32 (adler32-zlib) [ 8423.823748] Lustre: Lustre: Build Version: 2.17.57_103_g80273de [ 8424.087710] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 8425.799246] Key type lgssc registered [ 8427.472475] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8436.639440] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8436.667581] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8438.196702] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8443.537887] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 8448.488296] LustreError: 270376:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8448.513502] LustreError: 270376:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8453.179192] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8453.569086] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8458.488688] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 8467.318773] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8474.005759] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8481.326594] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8481.657991] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8484.741404] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8487.713545] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 8496.241157] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8497.164982] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8500.202514] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8500.211489] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8500.240270] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8504.296708] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8504.298238] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8504.298496] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8504.328168] Lustre: Skipped 1 previous similar message [ 8505.613987] Lustre: server umount lustre-OST0000 complete [ 8514.529588] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8514.531866] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8514.547982] Lustre: Skipped 1 previous similar message [ 8514.577051] Lustre: Skipped 1 previous similar message [ 8516.953318] Lustre: server umount lustre-MDT0000 complete [ 8519.655909] LustreError: 271368:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8519.689143] LustreError: 271368:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8520.346142] LustreError: 270356:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788783663 with bad export cookie 12934342180191147925 [ 8520.357121] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8520.382561] LustreError: 270356:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8520.728158] Lustre: server umount lustre-MDT0001 complete [ 8531.641758] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 8534.413643] Key type lgssc unregistered [ 8534.677530] LNet: 273961:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8534.682249] LNetError: 273961:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8534.697563] LNet: Removed LNI 192.168.202.148@tcp [ 8535.710278] Key type .llcrypt unregistered [ 8535.711988] Key type ._llcrypt unregistered [ 8555.579378] Key type ._llcrypt registered [ 8555.580942] Key type .llcrypt registered [ 8555.677744] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 8557.365422] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8557.395649] alg: No test for adler32 (adler32-zlib) [ 8558.570615] Lustre: Lustre: Build Version: 2.17.57_103_g80273de [ 8558.925155] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 8560.695583] Key type lgssc registered [ 8562.700207] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8570.663751] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8570.689165] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8586.463382] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8591.583349] LustreError: 275152:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.148@tcp: failed processing log, type 4: rc = -110 [ 8618.271468] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8625.745983] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 8634.737713] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8637.112694] Lustre: server umount lustre-OST0000 complete [ 8655.861340] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 8658.783825] Key type lgssc unregistered [ 8659.038450] LNet: 276784:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8659.061695] LNetError: 276784:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8659.091230] LNet: Removed LNI 192.168.202.148@tcp [ 8660.073780] Key type .llcrypt unregistered [ 8660.077610] Key type ._llcrypt unregistered [ 8703.366611] Key type ._llcrypt registered [ 8703.369440] Key type .llcrypt registered [ 8703.489631] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 8704.467902] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8704.508087] alg: No test for adler32 (adler32-zlib) [ 8705.507117] Lustre: Lustre: Build Version: 2.17.57_103_g80273de [ 8705.672986] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 8707.311168] Key type lgssc registered [ 8709.024713] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8723.585378] loop0: detected capacity change from 0 to 204800000 [ 8723.715329] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8723.725802] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8723.899284] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8725.059905] loop0: detected capacity change from 0 to 204800000 [ 8725.430380] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8725.470501] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8725.982303] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8727.165380] loop0: detected capacity change from 0 to 204800000 [ 8727.390367] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8727.404610] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8727.635885] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8729.340399] loop0: detected capacity change from 0 to 204800000 [ 8729.760348] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8729.784841] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8730.019881] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8731.882777] loop0: detected capacity change from 0 to 204800000 [ 8731.931464] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8738.321969] loop0: detected capacity change from 0 to 204800000 [ 8738.350482] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8742.839632] loop0: detected capacity change from 0 to 204800000 [ 8742.924824] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8747.819786] loop0: detected capacity change from 0 to 204800000 [ 8747.882097] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8752.418526] loop0: detected capacity change from 0 to 204800000 [ 8752.555824] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8752.604356] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8752.724301] Lustre: 280195:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8752.734712] Lustre: 280195:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8752.745978] Lustre: 280195:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8752.755260] Lustre: 280195:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8752.776157] Lustre: 280195:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8752.801315] Lustre: 280195:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8752.818616] Lustre: 280195:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8752.831850] Lustre: 280195:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8752.849105] Lustre: 280195:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8752.867733] Lustre: 280195:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8752.881664] Lustre: 280195:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8752.896757] Lustre: 280195:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8752.906716] Lustre: 280195:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8752.917876] Lustre: 280195:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8758.134459] Lustre: server umount MGS complete [ 8758.818255] loop0: detected capacity change from 0 to 204800000 [ 8758.910828] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8759.597983] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8759.643352] Lustre: lustre-MDT0000: Aborting client recovery [ 8760.568327] loop1: detected capacity change from 0 to 204800000 [ 8760.699464] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8760.743817] Lustre: 280629:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8760.750319] Lustre: 280629:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8760.757394] Lustre: 280629:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8760.764824] Lustre: 280629:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8760.771547] Lustre: 280629:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8760.779385] Lustre: 280629:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8760.787385] Lustre: 280629:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8760.803904] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8760.945782] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8761.007931] Lustre: lustre-MDT0001: Aborting client recovery [ 8762.511180] loop2: detected capacity change from 0 to 204800000 [ 8762.699914] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8762.816805] Lustre: 280708:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8762.841740] Lustre: 280708:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8762.860973] Lustre: 280708:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8762.874398] Lustre: 280708:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8762.906452] Lustre: 280708:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8762.932971] Lustre: 280708:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8762.970040] Lustre: 280708:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8763.116696] Lustre: Found index 0 for lustre-OST0000, updating log [ 8763.174290] Lustre: 280725:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8763.218732] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8763.239366] Lustre: lustre-OST0000: Aborting recovery [ 8764.084503] loop3: detected capacity change from 0 to 204800000 [ 8764.162438] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8764.197574] Lustre: 280790:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8764.208450] Lustre: 280790:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8764.214461] Lustre: 280790:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8764.222939] Lustre: 280790:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8764.235996] Lustre: 280790:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8764.241884] Lustre: 280790:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8764.258173] Lustre: 280790:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8764.275854] Lustre: Found index 1 for lustre-OST0001, updating log [ 8764.294322] Lustre: 280796:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8764.411808] Lustre: lustre-OST0001: Aborting recovery [ 8769.537773] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8769.562724] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8769.584415] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8769.598287] LustreError: 280572:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 8769.637709] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8771.552915] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8771.554919] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8773.049500] LustreError: 280539:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1996us: rc = -2 [ 8773.584795] LustreError: 280540:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 901us: rc = -2 [ 8773.594829] LustreError: 280540:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 21 previous similar messages [ 8786.399724] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8786.414246] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8786.427114] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8786.914781] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8786.945921] Lustre: Skipped 2 previous similar messages [ 8786.952261] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8786.957776] Lustre: Skipped 2 previous similar messages [ 8792.040177] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8792.046544] Lustre: Skipped 3 previous similar messages [ 8792.561856] Lustre: server umount lustre-MDT0000 complete [ 8793.682257] LustreError: 280633:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788783936 with bad export cookie 8247951895469199602 [ 8793.693529] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8793.712658] LustreError: 280633:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8797.155372] LustreError: 281049:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8797.158462] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8797.174919] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8797.182380] LustreError: 281049:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8797.218147] Lustre: Skipped 1 previous similar message [ 8798.176263] LustreError: 280538:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8798.195070] LustreError: 280538:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8800.160122] Lustre: server umount lustre-MDT0001 complete [ 8807.399561] Lustre: server umount lustre-OST0000 complete [ 8814.992963] Lustre: server umount lustre-OST0001 complete [ 8823.249600] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 08:26:04 (1788783964) [ 8856.415713] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [ 8870.262675] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 8910.906791] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8911.386803] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8911.459940] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8911.575638] Lustre: lustre-MDT0000: new disk, initializing [ 8911.738706] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8911.744721] Lustre: Skipped 1 previous similar message [ 8911.760417] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8917.636472] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 8929.306702] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8929.465041] Lustre: 284863:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8929.571424] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8929.575820] Lustre: Skipped 1 previous similar message [ 8929.658735] Lustre: lustre-MDT0001: new disk, initializing [ 8929.766068] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8929.791991] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8929.797795] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8936.214922] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 8947.202672] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8954.316352] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8961.897755] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8962.344745] Lustre: lustre-OST0000: new disk, initializing [ 8962.363071] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8962.380599] Lustre: 286833:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8962.520208] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8969.303643] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 8971.856471] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8971.873529] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8972.028119] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8980.717634] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8987.747754] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8998.235933] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9002.636380] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9008.526507] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9054.180130] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9054.186720] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9054.209183] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9054.228128] Lustre: Skipped 4 previous similar messages [ 9056.220210] Lustre: server umount lustre-OST0000 complete [ 9062.884897] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9062.893154] LustreError: Skipped 1 previous similar message [ 9062.898462] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9062.909309] Lustre: Skipped 1 previous similar message [ 9062.923495] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9066.878204] LustreError: 288827:0:(ldlm_resource.c:1207:ldlm_resource_complain()) MGS: namespace resource [0x65727473756c:0x0:0x0].0x0 (0000000079a6e007) refcount nonzero (1) after lock cleanup; forcing cleanup. [ 9067.072631] Lustre: server umount lustre-MDT0000 complete [ 9069.538804] LustreError: 287092:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9069.553873] LustreError: 287092:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9070.686381] LustreError: 284856:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788784213 with bad export cookie 8247951895469202311 [ 9070.690785] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9070.714715] LustreError: 284856:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9071.143316] Lustre: server umount lustre-MDT0001 complete [ 9081.070805] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9086.587091] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 9091.862345] Lustre: server umount MGS complete [ 9103.997375] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9104.540340] LustreError: 290608:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9104.565785] LustreError: 290621:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9104.625796] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9109.667377] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 9109.993249] LustreError: 290609:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9119.203341] LustreError: 290608:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9119.221589] LustreError: 290608:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9119.452254] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9124.609172] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 9132.267497] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9138.774968] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9146.699652] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9147.199924] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9147.203103] Lustre: Skipped 1 previous similar message [ 9154.337285] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 9162.480778] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9190.376414] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9190.382934] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9190.397130] Lustre: Skipped 2 previous similar messages [ 9190.413175] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9190.426221] Lustre: Skipped 3 previous similar messages [ 9196.448946] Lustre: server umount lustre-OST0000 complete [ 9206.613694] Lustre: server umount lustre-MDT0000 complete [ 9210.751732] LustreError: 290590:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788784353 with bad export cookie 8247951895469204250 [ 9210.765167] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9211.298616] Lustre: server umount lustre-MDT0001 complete [ 9225.427578] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 9228.936695] Key type lgssc unregistered [ 9229.282050] LNet: 294552:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9229.285659] LNetError: 294552:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9229.297345] LNet: Removed LNI 192.168.202.148@tcp [ 9230.283568] Key type .llcrypt unregistered [ 9230.287751] Key type ._llcrypt unregistered [ 9268.996941] Key type ._llcrypt registered [ 9268.999873] Key type .llcrypt registered [ 9269.183399] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [ 9283.646735] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 9284.722589] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9284.770377] alg: No test for adler32 (adler32-zlib) [ 9286.010701] Lustre: Lustre: Build Version: 2.17.57_103_g80273de [ 9286.337951] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 9288.039173] Key type lgssc registered [ 9288.960392] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9325.481701] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9325.509371] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9326.969572] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9327.036701] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9327.195689] Lustre: lustre-MDT0000: new disk, initializing [ 9327.332381] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9327.361978] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9332.476103] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 9344.602074] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9344.797150] Lustre: 298508:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9344.911207] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9344.918961] Lustre: Skipped 1 previous similar message [ 9345.013256] Lustre: lustre-MDT0001: new disk, initializing [ 9345.113913] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9345.156190] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9345.165871] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9350.344483] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 9360.779766] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9366.922743] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9373.668578] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9373.962827] Lustre: lustre-OST0000: new disk, initializing [ 9373.967066] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9373.975824] Lustre: 300475:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9374.106212] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9381.496757] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 9383.507720] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9383.516695] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9383.648263] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9392.478799] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9399.505586] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9411.692815] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9421.415527] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9434.588398] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9476.071811] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9476.091140] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9476.091212] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9476.120937] Lustre: Skipped 1 previous similar message [ 9476.145683] Lustre: Skipped 1 previous similar message [ 9481.193214] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9481.206449] Lustre: Skipped 1 previous similar message [ 9486.316723] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9486.324621] Lustre: Skipped 1 previous similar message [ 9488.351958] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9488.490336] Lustre: server umount lustre-OST0000 complete [ 9493.473566] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9493.496603] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9493.530994] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9496.549633] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9501.664275] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9501.670914] Lustre: Skipped 3 previous similar messages [ 9507.295139] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9507.713256] Lustre: server umount lustre-MDT0000 complete [ 9512.312584] LustreError: 301331:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788784655 with bad export cookie 10684655909590656696 [ 9512.317803] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9512.318536] LustreError: 301331:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9512.964916] Lustre: server umount lustre-MDT0001 complete [ 9521.543376] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9526.761138] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 9531.497356] Lustre: server umount MGS complete [ 9543.276137] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9543.742527] LustreError: 304466:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9543.768733] LustreError: 304466:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9543.772896] LustreError: 304479:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9543.832607] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9548.664059] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 9549.285301] LustreError: 304467:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9553.413732] LustreError: 304466:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9556.678830] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9556.888204] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9560.514717] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 9568.569986] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9575.225608] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9583.085623] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9583.680135] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9590.845406] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 9600.803703] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9629.664537] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9629.672723] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9629.697715] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9629.704864] Lustre: Skipped 2 previous similar messages [ 9632.742649] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9635.827560] Lustre: server umount lustre-OST0000 complete [ 9642.978274] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9642.991971] Lustre: Skipped 2 previous similar messages [ 9648.096409] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9648.100184] Lustre: Skipped 5 previous similar messages [ 9654.239600] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9654.406581] Lustre: server umount lustre-MDT0000 complete [ 9658.336323] LustreError: 304466:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9658.347200] LustreError: 304466:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9658.550705] LustreError: 304447:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788784801 with bad export cookie 10684655909590658635 [ 9658.567722] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9658.571572] LustreError: 304447:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9658.919239] Lustre: server umount lustre-MDT0001 complete [ 9671.673513] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 9674.198183] Key type lgssc unregistered [ 9674.488883] LNet: 308411:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9674.498486] LNetError: 308411:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9674.506978] LNet: Removed LNI 192.168.202.148@tcp [ 9675.315124] Key type .llcrypt unregistered [ 9675.319098] Key type ._llcrypt unregistered [ 9700.209520] Key type ._llcrypt registered [ 9700.213343] Key type .llcrypt registered [ 9700.409546] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [ 9720.490632] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 9722.205809] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9722.263205] alg: No test for adler32 (adler32-zlib) [ 9723.426340] Lustre: Lustre: Build Version: 2.17.57_103_g80273de [ 9723.732802] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 9725.402111] Key type lgssc registered [ 9726.914727] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9753.752653] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 9771.727982] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9771.755924] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9773.058197] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9773.078909] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9773.129747] Lustre: lustre-MDT0000: new disk, initializing [ 9773.229891] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9773.247319] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9778.208890] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 9789.791185] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9790.002538] Lustre: lustre-OST0000: new disk, initializing [ 9790.004929] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9790.007487] Lustre: Skipped 1 previous similar message [ 9790.010990] Lustre: 312459:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9790.075825] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9791.637165] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9791.653098] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9791.687970] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9798.220083] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [10629.342685] Lustre: server umount lustre-MDT0000 complete [10633.952679] LustreError: 311645:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788785777 with bad export cookie 17778061503461023299 [10633.964659] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10644.679085] Lustre: server umount lustre-OST0000 complete [10673.218164] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [10683.950842] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [10724.244501] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10724.550503] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10724.578309] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10724.648975] Lustre: lustre-MDT0000: new disk, initializing [10724.734155] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10724.776786] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10729.145440] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [10739.998872] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10740.134837] Lustre: 316800:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10740.199849] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10740.204825] Lustre: Skipped 1 previous similar message [10740.283891] Lustre: lustre-MDT0001: new disk, initializing [10740.396395] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10740.454466] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10740.471969] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10745.190038] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [10756.282889] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10762.982844] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10770.763751] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10771.048190] Lustre: lustre-OST0000: new disk, initializing [10771.051665] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10771.058868] Lustre: 318768:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10771.135280] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10778.267768] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [10779.201060] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10779.215041] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10779.374631] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10788.952840] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10793.531268] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10793.827397] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10798.876294] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10799.137495] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10805.232903] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10805.238042] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10805.245864] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10805.258620] Lustre: Skipped 1 previous similar message [10807.361789] Lustre: server umount lustre-OST0000 complete [10812.384128] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10812.393949] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10812.412465] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10812.420277] Lustre: Skipped 1 previous similar message [10815.459394] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10815.465964] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10817.330419] Lustre: server umount lustre-MDT0000 complete [10821.293947] LustreError: 317594:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788785964 with bad export cookie 17778061503463964118 [10821.304391] LustreError: 317594:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10821.305191] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10821.811218] Lustre: server umount lustre-MDT0001 complete [10825.658195] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10827.701836] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 08:59:29 (1788785969) [10834.590295] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10835.100203] LustreError: 321082:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10835.114720] LustreError: 321082:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10835.180260] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10840.250496] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [10840.546044] LustreError: 321083:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10845.676114] LustreError: 321082:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10849.903088] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10850.595634] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10857.653277] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [10866.872572] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10874.879123] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10885.885775] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10886.361837] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10893.755630] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [10901.925499] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10910.118209] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10910.290263] Lustre: lustre-OST0001: new disk, initializing [10910.294332] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10910.303178] Lustre: 323919:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10910.387228] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10916.429485] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10916.437243] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10916.483703] LustreError: 324164:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10917.553342] LustreError: 324164:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10917.576557] LustreError: 324164:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10918.584344] LustreError: 324178:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10919.106395] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [10920.673229] LustreError: 324178:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10920.691246] LustreError: 324178:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [10924.850777] LustreError: 324178:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10924.873942] LustreError: 324178:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [10929.459332] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10929.765634] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10933.170569] LustreError: 324178:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10933.189619] LustreError: 324178:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [10937.967278] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff903dc4207800.ost_server_uuid 50 [10939.984740] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff903dc4207800.ost_server_uuid in FULL state after 0 sec [10946.235335] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff903dc4207800.ost_server_uuid 50 [10948.230593] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff903dc4207800.ost_server_uuid in IDLE state after 0 sec [10949.669870] LustreError: 324178:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10949.684143] LustreError: 324178:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 31 previous similar messages [10950.798412] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10965.691737] Lustre: lustre-OST0001: Not available for connect from 192.168.202.48@tcp (stopping) [10965.707718] Lustre: Skipped 1 previous similar message [10965.986298] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10965.998103] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10970.647120] Lustre: server umount lustre-OST0001 complete [10970.760814] LustreError: 325465:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.202.48@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10977.762248] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10977.762861] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10977.767784] LustreError: Skipped 3 previous similar messages [10977.772731] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10977.776513] Lustre: Skipped 2 previous similar messages [10977.798584] Lustre: Skipped 5 previous similar messages [10978.296456] LustreError: 323728:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10978.325279] LustreError: 323728:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10983.488459] Lustre: server umount lustre-OST0000 complete [10988.528052] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10988.542269] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10988.549601] Lustre: Skipped 1 previous similar message [10988.572165] Lustre: Skipped 3 previous similar messages [10994.296335] Lustre: server umount lustre-MDT0000 complete [10998.752855] LustreError: 321079:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10998.769189] LustreError: 321079:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10998.954165] LustreError: 321064:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788786142 with bad export cookie 17778061503463965000 [10998.960915] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10998.962284] LustreError: 321064:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10999.516908] Lustre: server umount lustre-MDT0001 complete [11012.117465] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [11015.112149] Key type lgssc unregistered [11015.465098] LNet: 326624:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11015.477436] LNetError: 326624:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11015.507848] LNet: Removed LNI 192.168.202.148@tcp [11016.705448] Key type .llcrypt unregistered [11016.708239] Key type ._llcrypt unregistered [11047.093642] Key type ._llcrypt registered [11047.095214] Key type .llcrypt registered [11047.206510] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [11048.576741] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11048.735616] alg: No test for adler32 (adler32-zlib) [11050.235896] Lustre: Lustre: Build Version: 2.17.57_103_g80273de [11051.051600] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [11052.879207] Key type lgssc registered [11054.185468] Lustre: Echo OBD driver; http://www.lustre.org/ [11063.274114] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11063.291149] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11065.075757] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11070.514614] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [11075.557055] LustreError: 328052:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11075.590379] LustreError: 328052:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11078.808211] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11080.675074] LustreError: 328053:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11085.795206] LustreError: 328052:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11087.045942] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11087.415909] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11092.955817] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [11103.037553] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11103.511034] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11110.702937] LustreError: 329735:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11110.709902] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [11110.895706] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [11118.052170] LustreError: 329733:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11118.069342] LustreError: 329733:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11119.750847] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11128.298819] LustreError: 329734:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11128.336199] LustreError: 329734:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11128.405879] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11128.722299] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11133.966643] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [11135.798953] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [11144.468712] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11152.881878] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [11153.134207] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11261.937079] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11261.953774] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11261.958186] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11267.051869] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11267.059051] Lustre: Skipped 2 previous similar messages [11272.163694] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11272.180672] Lustre: Skipped 1 previous similar message [11273.184411] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11273.463381] Lustre: server umount lustre-OST0001 complete [11277.285399] LustreError: 330077:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11277.304763] LustreError: 330077:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11281.911042] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11281.928709] LustreError: Skipped 1 previous similar message [11281.936956] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11281.957986] Lustre: Skipped 1 previous similar message [11281.983196] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11286.920976] Lustre: server umount lustre-OST0000 complete [11292.130934] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11292.148148] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11292.166912] Lustre: Skipped 1 previous similar message [11292.182704] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11292.196399] Lustre: Skipped 2 previous similar messages [11297.956347] Lustre: server umount lustre-MDT0000 complete [11303.308672] LustreError: 331737:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788786446 with bad export cookie 4366750780556967310 [11303.312890] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11303.321049] LustreError: 331737:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11303.894511] Lustre: server umount lustre-MDT0001 complete [11319.174358] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [11322.783975] Key type lgssc unregistered [11323.155873] LNet: 333002:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11323.167192] LNetError: 333002:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11323.185866] LNet: Removed LNI 192.168.202.148@tcp [11324.317231] Key type .llcrypt unregistered [11324.319483] Key type ._llcrypt unregistered [11378.435664] Key type ._llcrypt registered [11378.442956] Key type .llcrypt registered [11378.678933] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [11380.189271] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11380.301183] alg: No test for adler32 (adler32-zlib) [11381.573421] Lustre: Lustre: Build Version: 2.17.57_103_g80273de [11382.043055] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [11383.791224] Key type lgssc registered [11385.710884] Lustre: Echo OBD driver; http://www.lustre.org/ [11397.532228] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11397.542385] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11398.998158] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11403.827515] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [11409.379861] LustreError: 335288:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11409.410824] LustreError: 335288:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11413.453645] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11413.782650] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11418.797469] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [11423.063377] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11431.339056] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11431.601850] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11438.771758] LustreError: 337105:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11439.351726] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [11443.886268] LustreError: 336780:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11443.924529] LustreError: 336780:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11443.935463] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11449.318815] LustreError: 336780:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11449.339077] LustreError: 336780:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11450.151757] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11450.693861] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11456.040434] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11457.609519] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [11469.250736] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11477.307623] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11490.799758] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11490.805202] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11490.814927] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11491.812669] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11491.815062] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11491.826436] Lustre: Skipped 2 previous similar messages [11491.838028] Lustre: Skipped 3 previous similar messages [11496.938392] LustreError: 336287:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11496.970848] LustreError: 336287:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11498.788734] Lustre: server umount lustre-MDT0000 complete [11503.155580] LustreError: 335270:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788786646 with bad export cookie 16472286651366122220 [11503.158342] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11503.167260] LustreError: 335270:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11503.535914] Lustre: server umount lustre-MDT0001 complete [11517.092882] Lustre: server umount lustre-OST0000 complete [11532.483435] Lustre: server umount lustre-OST0001 complete [11555.190990] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [11566.855329] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11567.360024] LustreError: 340557:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11567.408795] LustreError: 340557:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [11567.462909] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11572.122719] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [11584.458122] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11585.131919] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11590.885423] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [11594.845729] Lustre: 341698:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11604.348248] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11604.792992] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11609.967328] LustreError: 342051:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11610.017479] LustreError: 342051:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11613.112703] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11613.968560] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [11624.073531] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11630.125643] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11634.977829] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [11645.096448] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11650.018898] Lustre: 343571:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11655.656035] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11655.670603] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11655.683956] Lustre: Skipped 3 previous similar messages [11655.696572] Lustre: Skipped 3 previous similar messages [11660.234976] Lustre: server umount lustre-MDT0000 complete [11660.785847] LustreError: 340557:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11660.803343] LustreError: 340557:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [11663.644969] LustreError: 340539:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788786806 with bad export cookie 16472286651366123928 [11663.658030] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11664.004339] Lustre: server umount lustre-MDT0001 complete [11679.407806] Lustre: server umount lustre-OST0000 complete [11682.080292] Lustre: 334551:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788786809/real 1788786809] req@ffff935686943b80 x1875678619363328/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788786825 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11682.100079] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11683.980550] Lustre: server umount lustre-OST0001 complete [11688.022669] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11689.835142] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11691.813085] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 09:13:53 (1788786833) [11718.140836] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [11728.587920] loop0: detected capacity change from 0 to 1099511627776 [11729.369168] SGI XFS with ACLs, security attributes, quota, no debug enabled [11729.383874] *** VALIDATE xfs *** [11729.433707] XFS (loop0): Mounting V5 Filesystem [11729.520040] XFS (loop0): Ending clean mount [11731.072358] loop1: detected capacity change from 0 to 18691697672192 [11840.493743] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [11856.943483] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [11900.068463] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11900.299452] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11900.314972] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11900.353909] Lustre: lustre-MDT0000: new disk, initializing [11900.463807] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11900.472987] Lustre: Skipped 1 previous similar message [11900.503625] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11907.646354] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [11922.951031] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11923.201404] Lustre: 348735:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11923.277373] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11923.280623] Lustre: Skipped 1 previous similar message [11923.340779] Lustre: lustre-MDT0001: new disk, initializing [11923.509898] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11923.526560] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11928.282021] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [11938.973562] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11945.773233] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11953.791992] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11954.175479] Lustre: lustre-OST0000: new disk, initializing [11954.190513] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11954.207346] Lustre: 350702:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11955.876983] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11955.897034] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11956.039387] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11961.076711] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [11971.855522] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11977.479699] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11977.621168] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11981.972229] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11982.364856] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11986.926644] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11986.928316] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11986.939602] LustreError: Skipped 1 previous similar message [11986.956456] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11986.975576] Lustre: Skipped 1 previous similar message [11987.021606] Lustre: Skipped 1 previous similar message [11991.594423] Lustre: server umount lustre-OST0000 complete [11997.153906] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11997.188406] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11997.201474] Lustre: Skipped 1 previous similar message [12003.503105] Lustre: server umount lustre-MDT0000 complete [12007.398859] LustreError: 349545:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12007.441350] LustreError: 349545:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12010.527681] LustreError: 349530:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788787153 with bad export cookie 16472286651366125475 [12010.554577] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12010.556808] LustreError: 349530:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [12010.892560] Lustre: server umount lustre-MDT0001 complete [12021.378650] XFS (loop0): Unmounting Filesystem [12026.951592] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 09:19:28 (1788787168) [12034.627212] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12035.331476] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12035.345434] Lustre: Skipped 2 previous similar messages [12040.255053] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [12049.072506] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12054.452583] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [12065.487957] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12075.185684] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12082.411949] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12087.151549] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [12093.775994] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12119.526454] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12119.533895] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12119.555536] Lustre: Skipped 1 previous similar message [12119.562114] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12119.564887] Lustre: Skipped 2 previous similar messages [12123.831842] Lustre: server umount lustre-OST0000 complete [12135.166963] Lustre: server umount lustre-MDT0000 complete [12135.905590] LustreError: 353263:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12135.917334] LustreError: 353263:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [12139.197806] LustreError: 353248:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788787282 with bad export cookie 16472286651366126357 [12139.207667] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12139.215815] LustreError: 353248:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12139.964365] Lustre: server umount lustre-MDT0001 complete [12150.827071] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [12153.579218] Key type lgssc unregistered [12153.999597] LNet: 356921:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12154.010665] LNetError: 356921:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12154.036159] LNet: Removed LNI 192.168.202.148@tcp [12155.325154] Key type .llcrypt unregistered [12155.327717] Key type ._llcrypt unregistered [12188.789341] Key type ._llcrypt registered [12188.791694] Key type .llcrypt registered [12188.872926] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [12190.284358] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12190.346806] alg: No test for adler32 (adler32-zlib) [12191.400912] Lustre: Lustre: Build Version: 2.17.57_103_g80273de [12191.711174] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [12193.353134] Key type lgssc registered [12195.347831] Lustre: Echo OBD driver; http://www.lustre.org/ [12204.051626] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12204.097880] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12205.794768] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12211.429693] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [12215.269536] LustreError: 358487:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12215.306555] LustreError: 358487:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12220.389840] LustreError: 358488:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12221.120149] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12221.689583] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12227.579932] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [12236.735788] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12242.731916] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12250.696946] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12251.078619] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12258.084466] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [12267.226948] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12285.941286] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12285.965916] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12289.706613] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.48@tcp (stopping) [12289.710896] Lustre: Skipped 1 previous similar message [12291.053633] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12291.066035] Lustre: Skipped 1 previous similar message [12292.036756] Lustre: server umount lustre-MDT0001 complete [12294.787765] LustreError: 359921:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.48@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12299.899027] LustreError: 360985:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.48@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12299.929522] LustreError: 360985:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12304.602189] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12304.749946] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12304.754834] Lustre: Found index 1 for lustre-MDT0001, updating log [12304.801706] Lustre: 358476:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12305.251561] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12310.529733] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12311.500273] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [12311.561246] LustreError: 361959:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12311.578759] LustreError: 361959:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12311.588627] LustreError: 358480:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12311.681853] LustreError: 361966:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.148@tcp: cfg command failed: rc = -17 [12311.696782] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [12311.696782] [12325.522039] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.48@tcp (stopping) [12325.859389] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12325.863137] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12325.895081] Lustre: Skipped 1 previous similar message [12327.585524] Lustre: server umount lustre-MDT0001 complete [12330.617751] LustreError: 358482:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.48@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12330.653915] LustreError: 358482:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12340.533400] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12340.644260] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12340.655252] Lustre: Found index 1 for lustre-MDT0001, updating log [12340.758323] Lustre: 358476:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12340.854175] LustreError: 360985:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.48@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12340.874405] LustreError: 360985:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [12341.176930] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12346.358493] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12347.033856] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [12347.440645] LustreError: 363050:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12347.452129] LustreError: 363050:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12347.460918] LustreError: 358480:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12347.463173] LustreError: 358480:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12347.515950] LustreError: 363057:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12347.529970] LustreError: 363057:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12347.541721] LustreError: 363057:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.148@tcp: cfg command failed: rc = -17 [12347.554137] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.148@tcp [12347.554137] [12658.330189] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.48@tcp (stopping) [12658.340054] Lustre: Skipped 2 previous similar messages [12658.663139] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12658.664170] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12658.714583] Lustre: Skipped 2 previous similar messages [12660.098205] Lustre: server umount lustre-MDT0001 complete [12663.417769] LustreError: 359921:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.48@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12671.154473] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12671.250377] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12671.258379] Lustre: Found index 1 for lustre-MDT0001, updating log [12671.311200] Lustre: 358476:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12671.703866] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12676.414361] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [12677.104410] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12681.712288] LustreError: 371518:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12681.720395] LustreError: 371518:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12681.726611] LustreError: 358480:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12681.738320] LustreError: 358480:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12681.815065] LustreError: 371521:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12681.834958] LustreError: 371521:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.148@tcp: cfg command failed: rc = -114 [12681.847517] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12681.847517] [13004.788674] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13004.799417] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13004.802931] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13004.819313] Lustre: Skipped 2 previous similar messages [13004.835457] Lustre: Skipped 4 previous similar messages [13007.105642] Lustre: server umount lustre-MDT0000 complete [13009.888561] LustreError: 358484:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13009.919867] LustreError: 358484:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [13011.467127] LustreError: 361694:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788788154 with bad export cookie 14765117243046887851 [13011.474405] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13011.486027] LustreError: 361694:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [13015.012355] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13018.235877] Lustre: server umount lustre-MDT0001 complete [13022.595241] Lustre: server umount lustre-OST0000 complete [13031.486585] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [13040.396880] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [13075.886879] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 09:36:57 (1788788217) [13081.877895] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13082.110632] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13082.138996] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13082.222332] Lustre: lustre-MDT0000: new disk, initializing [13082.345224] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13082.361463] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13087.075992] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [13099.781576] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13099.984573] Lustre: 382291:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13100.127425] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13100.140075] Lustre: Skipped 1 previous similar message [13100.311074] Lustre: lustre-MDT0001: new disk, initializing [13100.459441] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13100.527264] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13100.562488] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13106.232415] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [13116.268206] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13121.478884] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13128.175489] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13128.465972] Lustre: lustre-OST0000: new disk, initializing [13128.468810] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13128.475770] Lustre: 384260:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13128.569301] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13134.447732] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [13135.448696] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13135.465825] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13135.638713] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13143.853418] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13151.207525] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13151.223117] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13151.238351] Lustre: Skipped 1 previous similar message [13151.248263] Lustre: Skipped 2 previous similar messages [13152.519569] Lustre: server umount lustre-MDT0000 complete [13155.963280] LustreError: 382282:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788788299 with bad export cookie 14765117243046889804 [13155.975072] LustreError: 382282:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [13155.975720] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13156.347360] Lustre: server umount lustre-MDT0001 complete [13170.267270] Lustre: server umount lustre-OST0000 complete [13200.975502] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [13211.194747] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [13241.249172] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12876 sec ========= 09:39:43 (1788788383) [13242.629889] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 09:39:44 (1788788384) === [13245.690416] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 09:39:47 (1788788387) === [13283.076434] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [13286.509882] Key type lgssc unregistered [13286.783523] LNet: 390622:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13286.795914] LNetError: 390622:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13286.817592] LNet: Removed LNI 192.168.202.148@tcp [13287.747421] Key type .llcrypt unregistered [13287.749540] Key type ._llcrypt unregistered