[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 490417419 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.002243] x2apic enabled [ 0.003006] Switched APIC routing to physical x2apic. [ 0.004007] kvm-guest: setup PV IPIs [ 0.006000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.006000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.006018] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.007008] pid_max: default: 32768 minimum: 301 [ 0.008117] LSM: Security Framework initializing [ 0.009034] Yama: becoming mindful. [ 0.010022] SELinux: Initializing. [ 0.010635] *** VALIDATE selinux *** [ 0.018173] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.022744] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.023131] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.024102] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.025125] *** VALIDATE tmpfs *** [ 0.027481] *** VALIDATE proc *** [ 0.029228] *** VALIDATE cgroup *** [ 0.030000] *** VALIDATE cgroup2 *** [ 0.030300] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.031165] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.033011] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.034029] Spectre V2 : User space: Vulnerable [ 0.035007] Speculative Store Bypass: Vulnerable [ 0.037459] debug: unmapping init [mem 0xffffffff97a59000-0xffffffff97a60fff] [ 0.039148] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.040593] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.041021] ... version: 2 [ 0.041872] ... bit width: 48 [ 0.042018] ... generic registers: 4 [ 0.043013] ... value mask: 0000ffffffffffff [ 0.044017] ... max period: 00007fffffffffff [ 0.045017] ... fixed-purpose events: 3 [ 0.046013] ... event mask: 000000070000000f [ 0.047325] rcu: Hierarchical SRCU implementation. [ 0.049468] smp: Bringing up secondary CPUs ... [ 0.050588] x86: Booting SMP configuration: [ 0.051026] .... node #0, CPUs: #1 #2 #3 [ 0.054400] smp: Brought up 1 node, 4 CPUs [ 0.056014] smpboot: Max logical packages: 1 [ 0.057018] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.141788] node 0 deferred pages initialised in 81ms [ 0.145084] devtmpfs: initialized [ 0.146184] x86/mm: Memory block size: 128MB [ 0.148371] gcov: version magic: 0x41383552 [ 0.149573] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.154126] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.157330] pinctrl core: initialized pinctrl subsystem [ 0.160160] [ 0.160796] ************************************************************* [ 0.163013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.166013] ** ** [ 0.169014] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.172015] ** ** [ 0.174012] ** This means that this kernel is built to expose internal ** [ 0.177018] ** IOMMU data structures, which may compromise security on ** [ 0.180014] ** your system. ** [ 0.183013] ** ** [ 0.186013] ** If you see this message and you are not debugging the ** [ 0.189014] ** kernel, report this immediately to your vendor! ** [ 0.191013] ** ** [ 0.194014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.197012] ************************************************************* [ 0.200699] NET: Registered protocol family 16 [ 0.203455] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.206067] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.210074] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.214124] cpuidle: using governor menu [ 0.215953] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.219403] PCI: Using configuration type 1 for base access [ 0.222135] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.231119] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.233068] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.236057] cryptd: max_cpu_qlen set to 1000 [ 0.240201] ACPI: Added _OSI(Module Device) [ 0.241010] ACPI: Added _OSI(Processor Device) [ 0.241887] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.243008] ACPI: Added _OSI(Processor Aggregator Device) [ 0.246112] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.251292] ACPI: Interpreter enabled [ 0.253057] ACPI: PM: (supports S0 S3 S4 S5) [ 0.255018] ACPI: Using IOAPIC for interrupt routing [ 0.257123] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.261407] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.272445] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.275041] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.278022] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.282078] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.288416] acpiphp: Slot [2] registered [ 0.291126] acpiphp: Slot [5] registered [ 0.292123] acpiphp: Slot [6] registered [ 0.294110] acpiphp: Slot [7] registered [ 0.296141] acpiphp: Slot [8] registered [ 0.298101] acpiphp: Slot [9] registered [ 0.299104] acpiphp: Slot [10] registered [ 0.301110] acpiphp: Slot [3] registered [ 0.303122] acpiphp: Slot [4] registered [ 0.305113] acpiphp: Slot [11] registered [ 0.306087] acpiphp: Slot [12] registered [ 0.308141] acpiphp: Slot [13] registered [ 0.310280] acpiphp: Slot [14] registered [ 0.312095] acpiphp: Slot [15] registered [ 0.313156] acpiphp: Slot [16] registered [ 0.315112] acpiphp: Slot [17] registered [ 0.317180] acpiphp: Slot [18] registered [ 0.319116] acpiphp: Slot [19] registered [ 0.320108] acpiphp: Slot [20] registered [ 0.322130] acpiphp: Slot [21] registered [ 0.324115] acpiphp: Slot [22] registered [ 0.325107] acpiphp: Slot [23] registered [ 0.327126] acpiphp: Slot [24] registered [ 0.328138] acpiphp: Slot [25] registered [ 0.329087] acpiphp: Slot [26] registered [ 0.331096] acpiphp: Slot [27] registered [ 0.333163] acpiphp: Slot [28] registered [ 0.335147] acpiphp: Slot [29] registered [ 0.337126] acpiphp: Slot [30] registered [ 0.339146] acpiphp: Slot [31] registered [ 0.341072] PCI host bridge to bus 0000:00 [ 0.343026] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.346023] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.349024] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.352029] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.355031] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.358045] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.360265] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.364153] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.367473] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.376862] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.383945] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.387020] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.390025] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.393018] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.396342] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.399822] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.402062] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.406810] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.412015] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.424015] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.429014] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.435639] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.442017] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.448017] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.465017] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.476095] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.484016] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.493018] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.523019] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.534225] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.542017] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.555016] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.577020] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.586424] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.601017] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.610018] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.630019] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.642897] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.652025] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.661018] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.677015] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.684318] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.693018] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.702017] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.724018] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.739692] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.741303] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.743309] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.745284] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.746241] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.752038] iommu: Default domain type: Passthrough [ 0.754401] SCSI subsystem initialized [ 0.756099] ACPI: bus type USB registered [ 0.757090] usbcore: registered new interface driver usbfs [ 0.758051] usbcore: registered new interface driver hub [ 0.759145] usbcore: registered new device driver usb [ 0.761170] pps_core: LinuxPPS API ver. 1 registered [ 0.762007] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.764046] PTP clock support registered [ 0.765146] EDAC MC: Ver: 3.0.0 [ 0.766413] PCI: Using ACPI for IRQ routing [ 0.769192] NetLabel: Initializing [ 0.769895] NetLabel: domain hash size = 128 [ 0.770008] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.772059] NetLabel: unlabeled traffic allowed by default [ 0.773111] vgaarb: loaded [ 0.774353] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.775011] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.779228] clocksource: Switched to clocksource kvm-clock [ 0.887942] VFS: Disk quotas dquot_6.6.0 [ 0.889265] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.890818] *** VALIDATE ramfs *** [ 0.891772] *** VALIDATE hugetlbfs *** [ 0.892898] pnp: PnP ACPI init [ 0.894782] pnp: PnP ACPI: found 6 devices [ 0.912506] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.915588] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.917368] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.919326] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.920789] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.922434] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.925091] NET: Registered protocol family 2 [ 0.926902] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.930858] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.934497] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.939340] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.942795] TCP: Hash tables configured (established 65536 bind 65536) [ 0.945816] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.948960] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.951950] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.955065] NET: Registered protocol family 1 [ 0.957483] RPC: Registered named UNIX socket transport module. [ 0.959748] RPC: Registered udp transport module. [ 0.961513] RPC: Registered tcp transport module. [ 0.963305] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.965757] NET: Registered protocol family 44 [ 0.967342] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.969776] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.972142] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.974752] PCI: CLS 0 bytes, default 64 [ 0.976482] Unpacking initramfs... [ 2.240820] debug: unmapping init [mem 0xffff95493cc54000-0xffff95493ffbffff] [ 2.244401] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.246333] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.249274] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.718286] Initialise system trusted keyrings [ 2.719347] Key type blacklist registered [ 2.720589] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.732160] zbud: loaded [ 2.736055] *** VALIDATE nfs *** [ 2.736883] *** VALIDATE nfs4 *** [ 2.739241] pstore: using deflate compression [ 2.741653] Platform Keyring initialized [ 2.841661] NET: Registered protocol family 38 [ 2.843489] Key type asymmetric registered [ 2.845306] Asymmetric key parser 'x509' registered [ 2.846537] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.848604] io scheduler mq-deadline registered [ 2.850264] io scheduler kyber registered [ 2.852138] io scheduler bfq registered [ 2.853585] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.855921] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.858404] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.861401] ACPI: Power Button [PWRF] [ 2.866847] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.873650] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.893091] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 2.900616] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 2.914707] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.943189] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.973033] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 2.978229] Non-volatile memory driver v1.3 [ 2.979356] Linux agpgart interface v0.103 [ 3.012255] virtio_blk virtio1: [vda] 145792 512-byte logical blocks (74.6 MB/71.2 MiB) [ 3.015491] vda: detected capacity change from 0 to 74645504 [ 3.041186] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.044321] vdb: detected capacity change from 0 to 1073741824 [ 3.058795] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.060974] vdc: detected capacity change from 0 to 2621440000 [ 3.075810] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.077928] vdd: detected capacity change from 0 to 2621440000 [ 3.092949] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.095859] vde: detected capacity change from 0 to 4294967296 [ 3.111534] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.114657] vdf: detected capacity change from 0 to 4294967296 [ 3.121653] libphy: Fixed MDIO Bus: probed [ 3.127726] usbcore: registered new interface driver usbserial_generic [ 3.130572] usbserial: USB Serial support registered for generic [ 3.133287] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.138643] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.140666] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.143461] mousedev: PS/2 mouse device common for all mice [ 3.146396] rtc_cmos 00:05: RTC can wake from S4 [ 3.150315] rtc_cmos 00:05: registered as rtc0 [ 3.152353] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.155438] intel_pstate: CPU model not supported [ 3.158659] hid: raw HID events driver (C) Jiri Kosina [ 3.161017] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.161076] usbcore: registered new interface driver usbhid [ 3.166132] usbhid: USB HID core driver [ 3.167758] drop_monitor: Initializing network drop monitor service [ 3.168088] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.170264] Initializing XFRM netlink socket [ 3.176055] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.176080] NET: Registered protocol family 10 [ 3.182627] Segment Routing with IPv6 [ 3.184204] NET: Registered protocol family 17 [ 3.186421] mpls_gso: MPLS GSO support [ 3.192450] RAS: Correctable Errors collector initialized. [ 3.194298] AVX version of gcm_enc/dec engaged. [ 3.196036] AES CTR mode by8 optimization enabled [ 3.268807] sched_clock: Marking stable (3268776846, 0)->(4022664395, -753887549) [ 3.271727] registered taskstats version 1 [ 3.273263] Loading compiled-in X.509 certificates [ 3.275279] zswap: loaded using pool lzo/zbud [ 3.300052] Key type big_key registered [ 3.311194] Key type encrypted registered [ 3.312833] ima: No TPM chip found, activating TPM-bypass! [ 3.314813] ima: Allocated hash algorithm: sha1 [ 3.316683] ima: No architecture policies found [ 3.318380] evm: Initialising EVM extended attributes: [ 3.319700] evm: security.selinux [ 3.320923] evm: security.ima [ 3.322015] evm: security.capability [ 3.323643] evm: HMAC attrs: 0x1 [ 3.325794] rtc_cmos 00:05: setting system clock to 2026-07-16 07:27:17 UTC (1784186837) [ 3.330966] debug: unmapping init [mem 0xffffffff98a03000-0xffffffff98bfffff] [ 3.334023] debug: unmapping init [mem 0xffffffff97782000-0xffffffff97a58fff] [ 3.342086] Write protecting the kernel read-only data: 28672k [ 3.345155] debug: unmapping init [mem 0xffffffff95e03000-0xffffffff95ffffff] [ 3.346957] debug: unmapping init [mem 0xffffffff96714000-0xffffffff967fffff] [ 3.379383] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.387930] systemd[1]: Detected virtualization kvm. [ 3.389876] systemd[1]: Detected architecture x86-64. [ 3.391796] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.418186] systemd[1]: No hostname configured. [ 3.419755] systemd[1]: Set hostname to . [ 3.422136] random: systemd: uninitialized urandom read (16 bytes read) [ 3.424738] systemd[1]: Initializing machine ID from random generator. [ 3.463460] random: ln: uninitialized urandom read (6 bytes read) [ 3.532760] random: systemd: uninitialized urandom read (16 bytes read) [ 3.535444] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 3.539527] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ 3.545078] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Slices. [ OK ] Listening on Journal Socket. Starting Create Volatile Files and Directories... Starting Create list of required st…ce nodes for the current kernel... Starting Setup Virtual Console... Starting Journal Service... [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Timers. [ OK ] Reached target Initrd Root Device. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Started Memstrack Anylazing Service. Starting Apply Kernel Variables... [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.045898] device-mapper: uevent: version 1.0.3 [ 4.047911] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 4.817852] virtio_net virtio0 ens2: renamed from eth0 [ 4.825798] random: fast init done [ 4.877377] scsi host0: ata_piix [ 4.901346] scsi host1: ata_piix [ 4.902491] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 4.904237] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.327962] dracut-initqueue[595]: RTNETLINK answers: File exists [ 9.828746] random: crng init done [ 9.829878] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.239731] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Initrd Default Target. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Swap. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.355900] printk: systemd: 26 output lines suppressed due to ratelimiting [ 11.575602] SELinux: Disabled at runtime. [ 11.632844] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.637803] systemd[1]: Detected virtualization kvm. [ 11.638848] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.151818] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.154653] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.159135] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.163754] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.168717] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.181729] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.193104] systemd[1]: Mounting Huge Pages File System... Mounting Huge Pages File System... [ OK ] Listening on Process Core Dump Socket. [ OK ] Reached target rpc_pipefs.target. [ OK ] Started Dispatch Password Requests to Console Directory Watch. Mounting Kernel Debug File System... Starting Remount Root and Kernel File Systems... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on udev Kernel Socket. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on udev Control Socket. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached targ[ 12.314905] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS et Paths. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. Starting udev Coldplug all Devices... Starting Apply Kernel Variables... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Stopped target Initrd Root File System. [ OK ] Created slice system-getty.slice. Mounting POSIX Message Queue File System... [ OK ] Started Journal Service. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted Kernel Debug File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Configure read-only root support... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Mounted /mnt. [ OK ] Started udev Coldplug all Devices. [ 12.724319] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.057050] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.113774] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.237902] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.248731] EDAC sbridge: Ver: 1.1.2 [ 14.958181] Key type dns_resolver registered [ 15.287125] NFS: Registering the id_resolver key type [ 15.288593] Key type id_resolver registered [ 15.289656] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Mark the need to relabel after reboot... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started RPC Bind. [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started D-Bus System Message Bus. Starting Login Service... Starting Network Manager... [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Started irqbalance daemon. [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg108-server login: [ 70.176138] hrtimer: interrupt took 11120740 ns [ 77.994322] libcfs: loading out-of-tree module taints kernel. [ 78.075964] Key type ._llcrypt registered [ 78.078222] Key type .llcrypt registered [ 78.200969] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_hostid [ 100.852228] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 103.398154] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 103.430178] alg: No test for adler32 (adler32-zlib) [ 105.240499] Lustre: Lustre: Build Version: 2.17.54_162_g9958c38 [ 106.631102] LNet: Added LNI 192.168.201.108@tcp [8/256/0/180] [ 108.535426] Key type lgssc registered [ 111.317102] Lustre: Echo OBD driver; http://www.lustre.org/ [ 136.897784] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 190.151929] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 205.679773] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 205.754865] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 207.201255] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 207.310313] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 207.525589] Lustre: lustre-MDT0000: new disk, initializing [ 207.664065] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 207.684981] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 212.885929] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 228.391715] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 228.566115] Lustre: 6534:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 228.609886] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 228.619322] Lustre: Skipped 1 previous similar message [ 228.761524] Lustre: lustre-MDT0001: new disk, initializing [ 228.884676] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 228.930379] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 228.952842] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 234.477518] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 239.753500] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 250.368299] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 250.581314] Lustre: lustre-OST0000: new disk, initializing [ 250.585738] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 250.591731] Lustre: 8471:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 250.638750] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 257.107531] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 257.125326] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 257.258667] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 258.229365] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 275.788630] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 275.980856] Lustre: lustre-OST0001: new disk, initializing [ 275.996982] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 276.008229] Lustre: 9544:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 276.105315] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 282.671314] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 282.678217] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 282.776024] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 283.991530] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 297.545598] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 305.794427] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 313.170341] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing check_logdir /tmp/testlogs/ [ 319.308565] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing yml_node [ 324.269747] Lustre: DEBUG MARKER: Client: 2.17.54.162 [ 327.144842] Lustre: DEBUG MARKER: MDS: 2.17.54.162 [ 329.760589] Lustre: DEBUG MARKER: OSS: 2.17.54.162 [ 331.632458] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Thu Jul 16 03:32:44 EDT 2026 [ 351.638675] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 353.310195] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 372.192606] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 372.202798] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 372.219117] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 374.752770] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 374.753036] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 374.758623] Lustre: Skipped 1 previous similar message [ 374.779922] Lustre: Skipped 1 previous similar message [ 376.712700] Lustre: server umount lustre-MDT0000 complete [ 380.949093] LustreError: 7481:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784187215 with bad export cookie 8074527953558171377 [ 380.956218] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 380.970531] LustreError: 7481:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 384.993279] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 385.003786] Lustre: Skipped 1 previous similar message [ 385.008037] LustreError: 6539:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 385.014440] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 385.014448] Lustre: Skipped 2 previous similar messages [ 385.051267] LustreError: 6539:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 386.016980] LustreError: 8769:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 386.037219] LustreError: 8769:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 387.462808] Lustre: server umount lustre-MDT0001 complete [ 392.157783] Lustre: server umount lustre-OST0000 complete [ 396.578260] Lustre: server umount lustre-OST0001 complete [ 403.662379] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_hostid [ 413.555668] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 455.502499] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 455.886640] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 455.933684] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 456.080206] Lustre: lustre-MDT0000: new disk, initializing [ 456.155881] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 456.183194] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 460.569173] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 474.415081] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 474.493794] Lustre: 15097:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 474.559947] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 474.564956] Lustre: Skipped 1 previous similar message [ 474.713185] Lustre: lustre-MDT0001: new disk, initializing [ 474.902957] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 474.977518] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 474.988111] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 480.584404] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 493.601300] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 501.585903] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 509.489071] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 509.865816] Lustre: lustre-OST0000: new disk, initializing [ 509.877318] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 509.888829] Lustre: 17068:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 510.008742] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 511.074894] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 511.103892] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 511.280663] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 517.346420] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 528.106762] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 533.681688] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 534.030891] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 539.515366] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 539.822300] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 542.180719] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 542.185906] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 542.193294] LustreError: Skipped 1 previous similar message [ 542.198175] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 542.198182] Lustre: Skipped 3 previous similar messages [ 542.257758] Lustre: Skipped 2 previous similar messages [ 547.299124] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 547.305672] Lustre: Skipped 1 previous similar message [ 548.671556] Lustre: server umount lustre-OST0000 complete [ 557.040379] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 557.058203] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 557.093507] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 557.106406] Lustre: Skipped 1 previous similar message [ 568.288567] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 568.584955] Lustre: server umount lustre-MDT0000 complete [ 571.878170] LustreError: 15104:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 571.890853] LustreError: 15104:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 573.007369] LustreError: 15089:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784187407 with bad export cookie 8074527953558172917 [ 573.019437] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 573.025927] LustreError: 15089:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 573.441535] Lustre: server umount lustre-MDT0001 complete [ 578.312742] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 03:36:51 (1784187411) [ 586.290876] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 587.032305] LustreError: 19334:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 587.154576] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 592.364511] LustreError: 19335:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 592.841438] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 602.594624] LustreError: 19334:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 602.609062] LustreError: 19334:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 603.678105] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 610.822972] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 619.507107] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 626.486851] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 634.151357] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 641.821750] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 649.881457] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 666.230260] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 675.809286] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 675.827078] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 675.841105] Lustre: Skipped 1 previous similar message [ 675.849478] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 675.852306] Lustre: Skipped 5 previous similar messages [ 680.851970] Lustre: server umount lustre-MDT0000 complete [ 682.999412] LustreError: 19335:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 683.015325] LustreError: 19335:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 685.283173] LustreError: 19315:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784187519 with bad export cookie 8074527953558173799 [ 685.291244] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 685.304342] LustreError: 19315:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 685.742524] Lustre: server umount lustre-MDT0001 complete [ 699.787770] Lustre: server umount lustre-OST0000 complete [ 719.597727] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 731.024070] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 731.409614] LustreError: 23743:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 731.564137] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 731.567341] Lustre: Skipped 2 previous similar messages [ 736.983680] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 746.511193] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 751.814963] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 756.108361] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 763.412553] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 770.258851] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 780.223774] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 780.454440] Lustre: lustre-OST0001: new disk, initializing [ 780.460853] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 780.471513] Lustre: 26058:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 782.333139] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 782.347170] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 782.464322] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 788.547108] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 800.031550] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 804.655442] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 822.241708] Lustre: server umount lustre-MDT0000 complete [ 823.284560] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 823.295530] LustreError: 23744:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 823.308365] Lustre: Skipped 4 previous similar messages [ 823.344247] LustreError: 23744:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 827.538964] LustreError: 23724:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784187661 with bad export cookie 8074527953558174646 [ 827.542090] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 827.552523] LustreError: 23724:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 828.078436] Lustre: server umount lustre-MDT0001 complete [ 843.237495] Lustre: server umount lustre-OST0000 complete [ 844.133092] Lustre: 3665:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1784187662/real 1784187662] req@ffff954884cb2a00 x1870855605059328/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1784187678 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 847.539771] Lustre: server umount lustre-OST0001 complete [ 870.491484] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 882.950110] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 883.642589] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 883.651851] Lustre: Skipped 3 previous similar messages [ 889.162876] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 897.877138] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 898.019807] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 898.022554] Lustre: Skipped 3 previous similar messages [ 902.722432] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 906.167618] Lustre: 30231:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 915.038313] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 923.901753] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 934.539967] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 943.623712] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 953.440590] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 957.724599] Lustre: 32113:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 965.099324] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 965.118433] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 965.146033] Lustre: Skipped 2 previous similar messages [ 965.154726] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 971.005306] Lustre: server umount lustre-MDT0000 complete [ 971.247522] LustreError: 30612:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 971.278775] LustreError: 30612:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 11 previous similar messages [ 976.276231] LustreError: 29063:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784187810 with bad export cookie 8074527953558175941 [ 976.277724] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 976.354668] LustreError: 29063:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 977.037493] Lustre: server umount lustre-MDT0001 complete [ 982.419726] Lustre: server umount lustre-OST0000 complete [ 987.196246] Lustre: server umount lustre-OST0001 complete [ 997.625906] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 03:43:50 (1784187830) [ 1026.947387] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 1036.750541] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1042.097299] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 1052.634819] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1058.447622] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 1061.963910] Lustre: 36064:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1070.258156] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1078.969956] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 1089.519806] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1097.206875] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 1106.389143] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1110.378787] Lustre: 37942:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1115.621960] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1115.629267] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1115.632574] Lustre: Skipped 6 previous similar messages [ 1115.647842] Lustre: Skipped 9 previous similar messages [ 1121.788776] Lustre: server umount lustre-MDT0000 complete [ 1125.315124] LustreError: 34897:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784187959 with bad export cookie 8074527953558177047 [ 1125.318706] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1125.328118] LustreError: 34897:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1125.757415] Lustre: server umount lustre-MDT0001 complete [ 1140.242388] Lustre: server umount lustre-OST0000 complete [ 1142.240111] Lustre: 3664:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1784187960/real 1784187960] req@ffff95498a1bdf80 x1870855605165056/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1784187976 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1145.069121] Lustre: server umount lustre-OST0001 complete [ 1155.839981] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 03:46:29 (1784187989) [ 1174.431828] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 1185.899730] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1186.308079] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1186.317346] Lustre: Skipped 7 previous similar messages [ 1190.679925] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 1201.446067] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1209.013598] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 1213.180776] Lustre: 41237:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1223.729067] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1227.560928] LustreError: 41591:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1227.581537] LustreError: 41591:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 15 previous similar messages [ 1232.368238] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 1241.848977] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1250.587480] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 1258.111312] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1261.483775] Lustre: 43113:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1262.528624] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1262.538868] Lustre: Skipped 1 previous similar message [ 1267.681224] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1267.707623] Lustre: Skipped 5 previous similar messages [ 1271.997706] Lustre: server umount lustre-MDT0000 complete [ 1276.368680] LustreError: 40063:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784188110 with bad export cookie 8074527953558178153 [ 1276.377162] LustreError: 40063:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1276.382722] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1282.789331] Lustre: server umount lustre-MDT0001 complete [ 1292.055832] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1301.472156] LustreError: 3661:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff9549a7a6b800 x1870855605219328/t0(0) o250->MGC192.168.201.108@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1307.674039] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 1311.343039] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1317.493111] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1323.526614] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1323.542206] Lustre: Skipped 1 previous similar message [ 1323.737643] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 1336.022545] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1338.670669] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1348.287511] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1350.242880] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1358.818095] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1362.589422] Lustre: server umount lustre-MDT0000 complete [ 1366.209347] LustreError: 40065:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784188200 with bad export cookie 8074527953558179280 [ 1366.216407] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1366.223055] LustreError: 40065:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1366.529832] Lustre: server umount lustre-MDT0001 complete [ 1380.729723] Lustre: server umount lustre-OST0000 complete [ 1395.625231] Lustre: server umount lustre-OST0001 complete [ 1406.030515] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 03:50:39 (1784188239) [ 1423.305277] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 1437.611978] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1444.596628] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 1454.685862] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1460.434794] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 1463.962716] Lustre: 48879:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1471.375095] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1478.557467] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 1489.097873] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1496.421994] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 1504.592485] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1509.326451] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1524.326382] Lustre: server umount lustre-MDT0000 complete [ 1525.234402] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1525.262317] Lustre: Skipped 8 previous similar messages [ 1528.233275] LustreError: 47705:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784188362 with bad export cookie 8074527953558180071 [ 1528.262796] LustreError: 47705:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1528.263825] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1528.730297] Lustre: server umount lustre-MDT0001 complete [ 1543.430467] Lustre: server umount lustre-OST0000 complete [ 1546.207255] Lustre: 3662:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1784188364/real 1784188364] req@ffff95498430d180 x1870855605320320/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1784188380 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1547.745721] Lustre: server umount lustre-OST0001 complete [ 1559.962621] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 03:53:12 (1784188392) [ 1562.917365] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1565.155406] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 03:53:18 (1784188398) [ 1577.681205] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_hostid [ 1586.967422] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 1613.596043] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1614.058943] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1614.109496] Lustre: lustre-MDT0000: new disk, initializing [ 1614.172610] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1618.995823] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 1629.468400] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1639.086126] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1639.638819] Lustre: lustre-OST0000: new disk, initializing [ 1639.654523] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1639.674653] Lustre: Skipped 1 previous similar message [ 1639.704994] Lustre: 54962:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1641.713773] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1641.729639] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1641.771416] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1647.345314] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 1657.937751] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1660.534926] Lustre: DEBUG MARKER: create test files [ 1698.274667] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1698.298607] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1698.310242] Lustre: Skipped 19 previous similar messages [ 1699.418480] Lustre: server umount lustre-OST0000 complete [ 1704.088707] Lustre: server umount lustre-MDT0000 complete [ 1717.628564] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing unload_modules_local [ 1720.996162] Key type lgssc unregistered [ 1721.340982] LNet: 56594:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1721.346073] LNetError: 56594:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1721.362385] LNet: Removed LNI 192.168.201.108@tcp [ 1722.328323] Key type .llcrypt unregistered [ 1722.330382] Key type ._llcrypt unregistered [ 1758.082516] Key type ._llcrypt registered [ 1758.085216] Key type .llcrypt registered [ 1758.258497] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 1760.211523] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1760.235298] alg: No test for adler32 (adler32-zlib) [ 1761.346971] Lustre: Lustre: Build Version: 2.17.54_162_g9958c38 [ 1761.691864] LNet: Added LNI 192.168.201.108@tcp [8/256/0/180] [ 1763.383219] Key type lgssc registered [ 1764.851210] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1774.097118] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1774.129137] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1775.830274] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1781.587463] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 1790.683544] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1799.765869] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1800.191800] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1801.282022] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1801.425452] Lustre: 59334:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1801.436652] Lustre: 59334:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1801.449227] Lustre: 59334:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1801.461506] Lustre: 59334:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1801.471735] Lustre: 59334:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1801.491387] Lustre: 59334:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1808.122454] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 1816.710157] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1823.058954] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1829.333627] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1837.036094] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1837.036690] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1837.064419] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1839.266973] Lustre: server umount lustre-OST0000 complete [ 1843.212960] Lustre: server umount lustre-MDT0000 complete [ 1857.145459] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing unload_modules_local [ 1860.189698] Key type lgssc unregistered [ 1860.556495] LNet: 60944:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1860.567349] LNetError: 60944:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1860.582930] LNet: Removed LNI 192.168.201.108@tcp [ 1861.781184] Key type .llcrypt unregistered [ 1861.784497] Key type ._llcrypt unregistered [ 1897.751927] Key type ._llcrypt registered [ 1897.753207] Key type .llcrypt registered [ 1897.820874] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 1899.111248] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1899.159051] alg: No test for adler32 (adler32-zlib) [ 1900.230935] Lustre: Lustre: Build Version: 2.17.54_162_g9958c38 [ 1900.476059] LNet: Added LNI 192.168.201.108@tcp [8/256/0/180] [ 1902.191186] Key type lgssc registered [ 1903.753787] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1915.142609] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1915.188739] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1916.985409] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1923.440670] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 1933.104209] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1942.391575] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1942.823288] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1950.150883] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 1959.271314] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1962.489071] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1962.599861] Lustre: 63787:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1962.610448] Lustre: 63787:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1962.617412] Lustre: 63787:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1962.638748] Lustre: 63787:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1962.655610] Lustre: 63787:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1962.660401] Lustre: 63787:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1972.359436] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1988.088456] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1988.091129] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1988.484396] Lustre: server umount lustre-OST0000 complete [ 1994.060704] Lustre: server umount lustre-MDT0000 complete [ 2007.306037] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing unload_modules_local [ 2009.593529] Key type lgssc unregistered [ 2010.012681] LNet: 65378:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2010.032233] LNetError: 65378:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2011.112356] LNet: Removed LNI 192.168.201.108@tcp [ 2012.044784] Key type .llcrypt unregistered [ 2012.047833] Key type ._llcrypt unregistered [ 2035.593165] Key type ._llcrypt registered [ 2035.597673] Key type .llcrypt registered [ 2035.770441] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_hostid [ 2056.960995] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 2058.374819] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2058.511595] alg: No test for adler32 (adler32-zlib) [ 2059.794438] Lustre: Lustre: Build Version: 2.17.54_162_g9958c38 [ 2060.103096] LNet: Added LNI 192.168.201.108@tcp [8/256/0/180] [ 2061.887753] Key type lgssc registered [ 2063.430812] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2113.664038] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2113.731035] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2115.321468] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2115.406279] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2115.631687] Lustre: lustre-MDT0000: new disk, initializing [ 2115.913329] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2115.961621] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2123.441831] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 2135.761919] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2135.953342] Lustre: 68546:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2136.001779] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2136.004676] Lustre: Skipped 1 previous similar message [ 2136.073882] Lustre: lustre-MDT0001: new disk, initializing [ 2136.187172] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2136.224657] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2136.238079] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2141.862280] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 2153.896764] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2161.137295] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2169.533334] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2170.028596] Lustre: lustre-OST0000: new disk, initializing [ 2170.033530] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2170.045072] Lustre: 70519:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2170.151698] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2176.122285] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2176.137581] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2176.350065] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2178.735765] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 2190.416920] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2196.435830] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2196.806271] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2202.370404] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2202.666689] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2207.213652] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2207.225605] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2207.228220] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2207.232326] LustreError: Skipped 1 previous similar message [ 2207.253423] Lustre: Skipped 1 previous similar message [ 2211.368456] Lustre: server umount lustre-OST0000 complete [ 2217.447720] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2217.466803] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2217.473839] Lustre: Skipped 2 previous similar messages [ 2222.030808] Lustre: server umount lustre-MDT0000 complete [ 2226.052249] LustreError: 68539:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784189060 with bad export cookie 7630491260629873663 [ 2226.065778] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2226.071166] LustreError: 68539:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2226.411794] Lustre: server umount lustre-MDT0001 complete [ 2235.567409] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 04:04:28 (1784189068) [ 2247.946891] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 2297.175442] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_hostid [ 2307.894466] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 2351.435122] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2351.685452] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2351.713202] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2351.771357] Lustre: lustre-MDT0000: new disk, initializing [ 2351.846615] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2351.862990] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2357.364124] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 2370.210207] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2370.338434] Lustre: 76542:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2370.397754] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2370.401235] Lustre: Skipped 1 previous similar message [ 2370.483134] Lustre: lustre-MDT0001: new disk, initializing [ 2370.624133] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2370.699464] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2370.704438] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2376.012858] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 2387.328844] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2394.772349] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2403.165638] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2403.537364] Lustre: lustre-OST0000: new disk, initializing [ 2403.543443] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2403.553640] Lustre: 78509:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2403.698525] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2410.392638] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 2413.693186] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2413.716574] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2413.860493] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2422.555175] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2429.054561] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2429.440370] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2435.708647] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2436.024196] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2439.653310] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2439.658817] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2439.663057] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2439.680554] Lustre: Skipped 2 previous similar messages [ 2444.483602] Lustre: server umount lustre-OST0000 complete [ 2449.888282] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2449.891239] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2449.914440] Lustre: Skipped 1 previous similar message [ 2449.920258] Lustre: Skipped 1 previous similar message [ 2455.013159] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2455.022886] Lustre: Skipped 2 previous similar messages [ 2455.751484] Lustre: server umount lustre-MDT0000 complete [ 2459.113398] LustreError: 76553:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2459.141906] LustreError: 76553:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 2460.403612] LustreError: 76535:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784189294 with bad export cookie 7630491260629874545 [ 2460.410346] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2460.421618] LustreError: 76535:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2460.829725] Lustre: server umount lustre-MDT0001 complete [ 2470.559578] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 04:08:24 (1784189304) [ 2478.845522] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2479.393195] LustreError: 80921:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2479.479447] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2484.706345] LustreError: 80922:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2485.092402] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 2489.834816] LustreError: 80921:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2494.956494] LustreError: 80922:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2495.704770] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2501.267843] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 2509.524962] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2516.404234] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2524.999132] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2525.096791] LustreError: 82744:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2525.372764] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2525.385268] Lustre: Skipped 1 previous similar message [ 2533.751649] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 2543.119729] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2547.889766] Lustre: 83595:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting a17b3af2-2f82-4aff-b4ab-b0226df18173 at adminstrative request [ 2551.273690] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2551.285741] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2561.510198] Lustre: MGS: Client a17b3af2-2f82-4aff-b4ab-b0226df18173 (at 0@lo) reconnecting [ 2561.519127] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x69e4f35f9b90fadc to 0x69e4f35f9b90fd83 [ 2561.562296] Lustre: MGC192.168.201.108@tcp: Connection restored to 0@lo (at 0@lo) [ 2586.678164] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2586.915831] Lustre: lustre-OST0001: new disk, initializing [ 2586.919978] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2586.926053] Lustre: 83991:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2593.389591] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2593.405121] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2593.536264] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2594.708803] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 2605.433600] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2614.247437] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2614.261964] LustreError: Skipped 1 previous similar message [ 2614.271370] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2614.302931] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2615.608456] Lustre: server umount lustre-OST0001 complete [ 2619.364553] LustreError: 83608:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2619.393771] LustreError: 83608:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2622.944936] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2622.948166] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2622.955041] LustreError: Skipped 2 previous similar messages [ 2622.974109] Lustre: Skipped 1 previous similar message [ 2634.213571] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2634.462431] Lustre: server umount lustre-OST0000 complete [ 2639.329841] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2639.345948] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2639.356225] Lustre: Skipped 1 previous similar message [ 2639.388065] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2639.398318] Lustre: Skipped 7 previous similar messages [ 2644.540614] Lustre: server umount lustre-MDT0000 complete [ 2649.118021] LustreError: 80903:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784189483 with bad export cookie 7630491260629876099 [ 2649.129879] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2649.137198] LustreError: 80903:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2649.605811] Lustre: server umount lustre-MDT0001 complete [ 2661.483936] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 04:11:34 (1784189494) [ 2663.470841] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2665.154929] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 04:11:38 (1784189498) [ 2666.791365] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2668.973183] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 04:11:42 (1784189502) [ 2670.939300] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2673.250906] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 04:11:46 (1784189506) [ 2675.254249] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2677.848572] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 04:11:50 (1784189510) [ 2679.522606] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2686.300970] loop: module loaded [ 2686.319762] loop0: detected capacity change from 0 to 4294967296 [ 2686.360766] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2688.287970] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2692.267352] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2697.765266] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2730.446944] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2730.511320] LustreError: 88118:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2730.522184] LustreError: 88118:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2730.537438] LustreError: 88118:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2730.548376] LustreError: 88118:0:(super25.c:178:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2755.314769] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_hostid [ 2768.381354] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 2813.244791] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2813.626353] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2813.676752] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2813.855783] Lustre: lustre-MDT0000: new disk, initializing [ 2813.986380] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2813.993299] Lustre: Skipped 1 previous similar message [ 2814.035731] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2819.389902] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 2832.450672] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2832.665571] Lustre: 90993:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2833.078639] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2839.694375] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 2851.456110] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2857.732881] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2866.413924] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2866.804443] Lustre: 92962:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2874.922445] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 2876.597118] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2876.835940] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2885.186378] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2892.108790] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2892.393736] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2899.777342] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2900.207893] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2907.616362] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2907.622021] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2907.632379] LustreError: Skipped 1 previous similar message [ 2907.635077] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2907.652760] Lustre: Skipped 2 previous similar messages [ 2907.681847] Lustre: Skipped 2 previous similar messages [ 2917.346483] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2917.515646] Lustre: server umount lustre-OST0000 complete [ 2928.704180] Lustre: server umount lustre-MDT0000 complete [ 2933.234563] LustreError: 90998:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2933.272827] LustreError: 90998:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [ 2934.053141] LustreError: 90985:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784189768 with bad export cookie 7630491260629876715 [ 2934.055514] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2934.068625] LustreError: 90985:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 2934.934573] Lustre: server umount lustre-MDT0001 complete [ 2947.603856] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 04:16:20 (1784189780) [ 2957.092682] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2957.752502] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 2957.763525] Lustre: Skipped 2 previous similar messages [ 2964.410196] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 2974.880672] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2980.025894] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 2989.240481] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2996.071042] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3004.116567] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3013.373961] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 3023.946619] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3030.217534] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3030.393570] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3036.037184] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3036.279946] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3044.974410] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3045.304639] Lustre: lustre-OST0001: new disk, initializing [ 3045.317858] Lustre: Skipped 2 previous similar messages [ 3045.330612] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3045.340566] Lustre: Skipped 3 previous similar messages [ 3045.361359] Lustre: 98754:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3050.685869] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3050.701659] Lustre: Skipped 2 previous similar messages [ 3050.707515] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3050.799352] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3052.694068] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 3063.219854] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3069.208651] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 3069.477744] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3075.165615] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 3075.429318] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3097.321930] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 3122.647104] Lustre: Failing over lustre-MDT0000 [ 3122.666064] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3122.678770] Lustre: Skipped 2 previous similar messages [ 3122.686230] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3122.692652] Lustre: Skipped 9 previous similar messages [ 3123.149339] Lustre: server umount lustre-MDT0000 complete [ 3123.851059] LustreError: 100010:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.201.8@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3123.878472] LustreError: 100010:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 3143.139605] Lustre: 66771:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1784189961/real 1784189961] req@00000000e77f564f x1870857654008448/t0(0) o400->MGC192.168.201.108@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1784189977 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3143.176531] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3149.769838] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 3149.776819] LDISKFS-fs (dm-0): recovery complete [ 3149.793380] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3153.376139] LustreError: 66768:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000881daaa6 x1870857654015616/t0(0) o250->MGC192.168.201.108@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3153.757723] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 3154.515265] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 3154.518231] Lustre: 97426:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3158.457639] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 3159.020034] Lustre: 95380:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3159.022197] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3159.032587] Lustre: 95380:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 1 previous similar message [ 3159.104379] LustreError: 101603:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3159.440591] LustreError: 101603:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3159.769514] LustreError: 101603:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3159.798615] LustreError: 101603:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 3160.131104] LustreError: 101603:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3160.137447] LustreError: 101603:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 3160.146244] Lustre: 101603:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3160.170479] Lustre: 101603:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 7 previous similar messages [ 3160.826831] LustreError: 101603:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3160.835895] LustreError: 101603:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 2 previous similar messages [ 3161.159267] LustreError: 101603:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3161.167873] LustreError: 101603:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 2 previous similar messages [ 3162.457685] Lustre: 101603:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3162.481678] Lustre: 101603:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 6 previous similar messages [ 3163.123350] LustreError: 101603:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3163.144860] LustreError: 101603:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 6 previous similar messages [ 3163.471396] LustreError: 101603:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3163.479286] LustreError: 101603:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 6 previous similar messages [ 3166.672256] Lustre: 101603:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3166.689508] Lustre: 101603:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 12 previous similar messages [ 3167.360071] LustreError: 101603:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3167.369750] LustreError: 101603:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 3167.695157] LustreError: 101603:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3167.702773] LustreError: 101603:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 3174.725749] Lustre: 101603:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3174.731810] Lustre: 101603:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 24 previous similar messages [ 3175.370845] LustreError: 101603:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3175.387935] LustreError: 101603:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 24 previous similar messages [ 3175.711847] LustreError: 101603:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3175.725874] LustreError: 101603:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 24 previous similar messages [ 3190.960041] Lustre: 101603:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3190.979321] Lustre: 101603:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 50 previous similar messages [ 3191.642206] LustreError: 101603:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3191.662054] LustreError: 101603:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 50 previous similar messages [ 3191.983144] LustreError: 101603:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3191.992413] LustreError: 101603:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 50 previous similar messages [ 3214.271185] Lustre: 101603:0:(ldlm_lib.c:2393:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3214.284638] Lustre: 101603:0:(ldlm_lib.c:2403:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3214.294443] Lustre: 101603:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 977db420-3821-4052-868f-2be919570192@192.168.201.8@tcp [ 3214.304578] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3214.311338] Lustre: 101603:0:(ldlm_lib.c:1913:abort_req_replay_queue()) @@@ aborted: req@000000009cf94bf4 x1870857632812416/t0(8589934769) o101->977db420-3821-4052-868f-2be919570192@192.168.201.8@tcp:484/0 lens 592/0 e 0 to 0 dl 1784190059 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3214.340340] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3214.382145] Lustre: lustre-MDT0000: Denying connection for new client 977db420-3821-4052-868f-2be919570192 (at 192.168.201.8@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3214.412879] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3214.458569] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3214.461407] Lustre: Skipped 2 previous similar messages [ 3214.468728] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3214.514626] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:77 to 0x280000401:577) [ 3214.520134] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:45 to 0x2c0000401:545) [ 3230.177546] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3235.759586] Lustre: server umount lustre-OST0000 complete [ 3246.729167] Lustre: server umount lustre-OST0001 complete [ 3255.779574] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3255.789517] Lustre: Skipped 7 previous similar messages [ 3255.798647] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3255.805993] Lustre: Skipped 6 previous similar messages [ 3257.734228] Lustre: server umount lustre-MDT0000 complete [ 3261.408300] LustreError: 100010:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3261.426658] LustreError: 100010:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 41 previous similar messages [ 3262.657816] LustreError: 98753:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784190096 with bad export cookie 7630491260629927927 [ 3262.665773] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3262.670044] LustreError: 98753:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3263.045693] Lustre: server umount lustre-MDT0001 complete [ 3272.327566] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 04:21:45 (1784190105) [ 3280.643843] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3280.651176] Lustre: *** cfs_fail_loc=197, val=0*** [ 3296.287649] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3301.407912] LustreError: 103582:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.108@tcp: failed processing log, type 4: rc = -110 [ 3306.567686] Lustre: *** cfs_fail_loc=197, val=0*** [ 3306.573834] Lustre: Skipped 65 previous similar messages [ 3327.135793] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3327.143400] Lustre: Skipped 4 previous similar messages [ 3334.375670] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 3343.374797] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3345.982446] Lustre: server umount lustre-OST0000 complete [ 3355.273795] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 04:23:08 (1784190188) [ 3378.120085] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3384.438889] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 3395.623251] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3400.087314] LustreError: 105235:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3400.099717] LustreError: 105235:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3400.112413] LustreError: 105235:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 22, retries 0, failed: rc = -5 [ 3400.770385] Lustre: server umount lustre-MDT0000 complete [ 3439.391178] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_hostid [ 3448.527646] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 3488.853193] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3489.166750] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3489.195519] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3489.336529] Lustre: lustre-MDT0000: new disk, initializing [ 3489.503819] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3494.601097] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 3506.823885] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3506.919492] Lustre: 109415:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3507.151123] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3511.791215] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 3522.876504] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3530.341993] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3538.506424] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3538.862782] Lustre: 111383:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3545.958364] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 3547.404993] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3558.977699] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3565.550804] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3566.088523] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3571.845406] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3572.167936] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3578.342390] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3578.353447] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3578.361792] LustreError: Skipped 5 previous similar messages [ 3578.363092] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3578.389183] Lustre: Skipped 2 previous similar messages [ 3578.421226] Lustre: Skipped 3 previous similar messages [ 3580.845782] Lustre: server umount lustre-OST0000 complete [ 3592.300752] Lustre: server umount lustre-MDT0000 complete [ 3593.696522] LustreError: 110977:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3593.719304] LustreError: 110977:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 3596.566888] LustreError: 109406:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784190430 with bad export cookie 7630491260629938490 [ 3596.579377] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3596.591762] LustreError: 109406:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3597.164423] Lustre: server umount lustre-MDT0001 complete [ 3607.402137] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 04:27:20 (1784190440) [ 3618.266951] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing unload_modules_local [ 3621.498748] Key type lgssc unregistered [ 3621.849055] LNet: 113872:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3621.859068] LNetError: 113872:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3621.871470] LNet: Removed LNI 192.168.201.108@tcp [ 3622.894501] Key type .llcrypt unregistered [ 3622.898218] Key type ._llcrypt unregistered [ 3642.844714] Key type ._llcrypt registered [ 3642.847815] Key type .llcrypt registered [ 3642.963410] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_hostid [ 3657.392649] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 3659.129323] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3659.173444] alg: No test for adler32 (adler32-zlib) [ 3660.240696] Lustre: Lustre: Build Version: 2.17.54_162_g9958c38 [ 3660.660725] LNet: Added LNI 192.168.201.108@tcp [8/256/0/180] [ 3662.377051] Key type lgssc registered [ 3663.811909] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3714.956795] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3715.028870] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3716.454081] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3716.519577] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3716.612535] Lustre: lustre-MDT0000: new disk, initializing [ 3716.723627] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3716.746874] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3722.290095] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 3735.174460] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3735.579729] Lustre: lustre-OST0000: new disk, initializing [ 3735.591812] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3735.597153] Lustre: Skipped 1 previous similar message [ 3735.610953] Lustre: 118052:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3735.706076] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3743.879162] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 3745.464767] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3745.480597] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3745.517298] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3757.431110] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3757.598577] Lustre: lustre-OST0001: new disk, initializing [ 3757.603752] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3757.612490] Lustre: 118916:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3757.715354] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3765.683338] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 3766.501328] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3766.516520] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3766.564230] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3776.998306] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3777.025849] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3782.271512] Lustre: server umount lustre-MDT0000 complete [ 3789.911521] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3795.540599] LustreError: 117235:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784190629 with bad export cookie 17300187270698293900 [ 3795.556263] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3795.564298] LustreError: 117235:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3795.663439] Lustre: server umount lustre-OST0000 complete [ 3800.457492] Lustre: server umount lustre-OST0001 complete [ 3810.336196] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 04:30:43 (1784190643) [ 3841.561845] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 04:31:14 (1784190674) [ 3855.174829] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 3885.037086] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 04:31:58 (1784190718) [ 3933.083466] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_hostid [ 3946.708949] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 4002.641424] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4002.821694] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4002.865885] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4002.918724] Lustre: lustre-MDT0000: new disk, initializing [ 4003.007505] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4003.037889] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4010.287406] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 4024.914974] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4024.994925] Lustre: 126672:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4025.023409] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4025.025177] Lustre: Skipped 1 previous similar message [ 4025.086153] Lustre: lustre-MDT0001: new disk, initializing [ 4025.170936] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4025.208436] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4025.216295] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4031.164288] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 4043.005362] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4051.951315] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4060.542597] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4061.028167] Lustre: lustre-OST0000: new disk, initializing [ 4061.033861] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4061.070295] Lustre: 128651:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4061.255281] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4066.929865] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4066.954494] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4067.110713] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4069.492938] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 4081.557231] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4087.860657] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4088.206665] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4093.872350] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4094.142701] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4098.017577] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4098.026716] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4098.039825] Lustre: Skipped 1 previous similar message [ 4098.048391] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4098.058346] Lustre: Skipped 2 previous similar messages [ 4102.830568] Lustre: server umount lustre-OST0000 complete [ 4108.258909] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4108.282358] Lustre: Skipped 1 previous similar message [ 4108.296906] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4108.314869] Lustre: Skipped 1 previous similar message [ 4113.376440] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4113.391778] Lustre: Skipped 1 previous similar message [ 4114.126400] Lustre: server umount lustre-MDT0000 complete [ 4118.169951] LustreError: 126665:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784190952 with bad export cookie 17300187270698295125 [ 4118.186277] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4118.197398] LustreError: 126665:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4118.833781] Lustre: server umount lustre-MDT0001 complete [ 4130.598698] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 04:36:03 (1784190963) [ 4138.556768] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4139.115695] LustreError: 131064:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4139.131690] LustreError: 131064:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 4139.192081] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4144.635785] LustreError: 131065:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4145.461908] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 4149.729424] LustreError: 131064:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4154.854261] LustreError: 131065:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4156.591473] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4162.570392] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 4173.966330] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4182.819472] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4193.384810] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4193.984876] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4193.994515] Lustre: Skipped 1 previous similar message [ 4202.527953] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 4215.411616] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4221.414843] Lustre: *** cfs_fail_loc=159, val=0*** [ 4238.023569] Lustre: lustre-MDT0000: Client af6b00fb-1c86-4923-8da4-9b80734c7035 (at 192.168.201.8@tcp) reconnecting [ 4240.122993] Lustre: *** cfs_fail_loc=159, val=0*** [ 4240.134421] Lustre: Skipped 7 previous similar messages [ 4256.480481] Lustre: lustre-MDT0000: Client af6b00fb-1c86-4923-8da4-9b80734c7035 (at 192.168.201.8@tcp) reconnecting [ 4262.888712] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4262.904809] LustreError: Skipped 1 previous similar message [ 4262.907371] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4262.918690] Lustre: Skipped 1 previous similar message [ 4262.922677] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4264.220613] Lustre: server umount lustre-OST0000 complete [ 4269.554165] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4269.578420] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4271.073260] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4271.077202] Lustre: Skipped 1 previous similar message [ 4278.390455] Lustre: server umount lustre-MDT0000 complete [ 4281.316621] LustreError: 131064:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4281.355095] LustreError: 131064:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4283.522259] LustreError: 131044:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784191117 with bad export cookie 17300187270698296007 [ 4283.523173] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4283.534217] LustreError: 131044:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4284.010824] Lustre: server umount lustre-MDT0001 complete [ 4300.370292] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing unload_modules_local [ 4303.795287] Key type lgssc unregistered [ 4304.513695] LNet: 134843:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4304.539481] LNetError: 134843:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4304.592241] LNet: Removed LNI 192.168.201.108@tcp [ 4305.605759] Key type .llcrypt unregistered [ 4305.608733] Key type ._llcrypt unregistered [ 4338.498362] Key type ._llcrypt registered [ 4338.506047] Key type .llcrypt registered [ 4338.643611] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 4340.397937] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4340.518882] alg: No test for adler32 (adler32-zlib) [ 4341.810044] Lustre: Lustre: Build Version: 2.17.54_162_g9958c38 [ 4342.235877] LNet: Added LNI 192.168.201.108@tcp [8/256/0/180] [ 4344.008064] Key type lgssc registered [ 4345.420815] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4353.614694] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4353.652658] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4355.365675] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4361.976197] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 4365.796706] LustreError: 136259:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4365.823756] LustreError: 136259:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4370.912512] LustreError: 136260:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4373.282350] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4373.667319] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4379.608133] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 4389.406909] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4397.803341] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4406.596628] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4407.171509] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4415.868659] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 4425.201245] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4425.585116] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4436.024200] Lustre: *** cfs_fail_loc=159, val=0*** [ 4452.480920] Lustre: lustre-MDT0000: Client c0f25a35-9b91-4b54-bd36-31b4abf4e14c (at 192.168.201.8@tcp) reconnecting [ 4456.029321] Lustre: *** cfs_fail_loc=159, val=0*** [ 4472.497649] Lustre: lustre-MDT0001: Client c0f25a35-9b91-4b54-bd36-31b4abf4e14c (at 192.168.201.8@tcp) reconnecting [ 4474.749532] Lustre: *** cfs_fail_loc=159, val=0*** [ 4474.754755] Lustre: Skipped 5 previous similar messages [ 4490.441379] Lustre: lustre-MDT0001: Client c0f25a35-9b91-4b54-bd36-31b4abf4e14c (at 192.168.201.8@tcp) reconnecting [ 4495.865619] Lustre: 139298:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4499.889587] Lustre: *** cfs_fail_loc=159, val=0*** [ 4499.899122] Lustre: Skipped 2 previous similar messages [ 4515.476495] Lustre: lustre-MDT0000: Client 23a52736-f35c-4f65-b04e-a5059080b558 (at 192.168.201.8@tcp) reconnecting [ 4517.642119] Lustre: *** cfs_fail_loc=159, val=0*** [ 4517.651649] Lustre: Skipped 14 previous similar messages [ 4533.403556] Lustre: lustre-MDT0000: Client 23a52736-f35c-4f65-b04e-a5059080b558 (at 192.168.201.8@tcp) reconnecting [ 4535.592442] Lustre: 139591:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4543.472402] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4543.479773] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4543.503479] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4548.584605] LustreError: 139605:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4548.608347] LustreError: 139605:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4548.912247] Lustre: server umount lustre-OST0000 complete [ 4554.258579] Lustre: server umount lustre-MDT0000 complete [ 4557.807881] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4557.825265] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4557.842498] Lustre: Skipped 1 previous similar message [ 4557.857083] LustreError: 139359:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4559.019409] LustreError: 136239:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784191393 with bad export cookie 16726549433878092953 [ 4559.022619] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4559.028844] LustreError: 136239:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4559.575840] Lustre: server umount lustre-MDT0001 complete [ 4571.603608] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing unload_modules_local [ 4574.728377] Key type lgssc unregistered [ 4575.084821] LNet: 140649:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4575.090798] LNetError: 140649:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4575.102509] LNet: Removed LNI 192.168.201.108@tcp [ 4576.214994] Key type .llcrypt unregistered [ 4576.219727] Key type ._llcrypt unregistered [ 4603.846794] Key type ._llcrypt registered [ 4603.849607] Key type .llcrypt registered [ 4604.040649] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 4605.346845] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4605.513975] alg: No test for adler32 (adler32-zlib) [ 4606.737494] Lustre: Lustre: Build Version: 2.17.54_162_g9958c38 [ 4607.035807] LNet: Added LNI 192.168.201.108@tcp [8/256/0/180] [ 4608.759338] Key type lgssc registered [ 4610.455922] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4620.552864] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4620.570058] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4622.244246] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4629.219759] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 4632.550862] LustreError: 142066:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4632.586962] LustreError: 142066:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4633.578730] LustreError: 142067:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4638.694511] LustreError: 142066:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4640.586063] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4641.475621] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4648.099909] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 4656.016324] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4662.602983] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4669.874449] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4670.143420] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4675.788586] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 4684.503623] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4686.588285] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4693.761526] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4701.832985] Lustre: 144675:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4705.155297] Lustre: 144723:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4707.108378] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4708.431798] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4714.379897] Lustre: 144879:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4715.910914] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4717.056127] Lustre: 144978:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4722.657363] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4722.669232] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4722.681174] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4724.705813] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4724.712666] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4724.718882] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4727.567831] Lustre: server umount lustre-OST0000 complete [ 4733.414513] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4733.425479] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4733.441958] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4733.448132] Lustre: Skipped 1 previous similar message [ 4740.066732] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4740.085801] Lustre: Skipped 3 previous similar messages [ 4745.187331] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4747.231241] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4747.383579] Lustre: server umount lustre-MDT0000 complete [ 4750.308776] LustreError: 143909:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4750.318149] LustreError: 143909:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4752.372597] LustreError: 142047:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784191586 with bad export cookie 5022097748844710673 [ 4752.377789] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4752.396804] LustreError: 142047:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4753.015718] Lustre: server umount lustre-MDT0001 complete [ 4767.013984] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing unload_modules_local [ 4769.592720] Key type lgssc unregistered [ 4769.953905] LNet: 146037:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4769.962401] LNetError: 146037:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4769.988523] LNet: Removed LNI 192.168.201.108@tcp [ 4770.943801] Key type .llcrypt unregistered [ 4770.947434] Key type ._llcrypt unregistered [ 4798.348167] Key type ._llcrypt registered [ 4798.350277] Key type .llcrypt registered [ 4798.538073] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 4800.455871] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4800.495673] alg: No test for adler32 (adler32-zlib) [ 4801.624273] Lustre: Lustre: Build Version: 2.17.54_162_g9958c38 [ 4801.861254] LNet: Added LNI 192.168.201.108@tcp [8/256/0/180] [ 4803.559384] Key type lgssc registered [ 4805.184549] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4813.485458] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4813.515615] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4815.200261] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4820.263745] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 4825.583903] LustreError: 147454:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4825.612585] LustreError: 147454:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4829.489921] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4829.962820] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4833.956332] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 4841.472951] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4850.426886] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4860.495216] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4861.001373] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4869.399776] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 4875.318517] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4879.014357] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4885.567106] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4891.773388] Lustre: *** cfs_fail_loc=159, val=0*** [ 4907.165237] Lustre: lustre-MDT0000: Client bf35e2d9-b76e-467e-ac25-029e65219e80 (at 192.168.201.8@tcp) reconnecting [ 4926.437422] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4926.439052] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4926.451643] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4926.462850] Lustre: Skipped 1 previous similar message [ 4931.556915] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4931.565698] Lustre: Skipped 1 previous similar message [ 4931.836174] Lustre: server umount lustre-OST0000 complete [ 4936.673291] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4936.705514] Lustre: Skipped 1 previous similar message [ 4936.718893] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4941.793686] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4941.800051] Lustre: Skipped 2 previous similar messages [ 4942.587085] Lustre: server umount lustre-MDT0000 complete [ 4946.918086] LustreError: 150026:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4946.952860] LustreError: 150026:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4947.215284] LustreError: 150093:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784191781 with bad export cookie 9493606083513937624 [ 4947.230656] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4947.236906] LustreError: 150093:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4947.697713] Lustre: server umount lustre-MDT0001 complete [ 4961.501485] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing unload_modules_local [ 4965.011516] Key type lgssc unregistered [ 4965.386727] LNet: 151147:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4965.396182] LNetError: 151147:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4965.424845] LNet: Removed LNI 192.168.201.108@tcp [ 4966.772847] Key type .llcrypt unregistered [ 4966.777410] Key type ._llcrypt unregistered [ 4992.693382] Key type ._llcrypt registered [ 4992.697768] Key type .llcrypt registered [ 4992.833711] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 4993.892628] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4994.061409] alg: No test for adler32 (adler32-zlib) [ 4995.171435] Lustre: Lustre: Build Version: 2.17.54_162_g9958c38 [ 4995.455567] LNet: Added LNI 192.168.201.108@tcp [8/256/0/180] [ 4997.154973] Key type lgssc registered [ 4998.271773] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5006.948846] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5006.985579] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5008.745506] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5014.731250] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 5018.090337] LustreError: 152564:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5018.110098] LustreError: 152564:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5023.215823] LustreError: 152565:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5026.704139] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5027.240791] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5031.254550] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 5039.705039] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5046.752741] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5054.616629] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5055.096724] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5061.961063] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 5071.666521] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5072.536379] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 5078.731153] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 5103.887806] Lustre: lustre-MDT0001: evicting 426335b8-1c63-4777-a43a-e415d72a21fe (at 192.168.201.8@tcp) by administrative request [ 5103.896091] Lustre: Skipped 1 previous similar message [ 5134.304820] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5134.311774] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5134.344200] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5135.335813] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5135.338677] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5135.366788] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5140.145455] Lustre: server umount lustre-OST0000 complete [ 5150.176352] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5150.181066] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5150.198399] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5150.214269] Lustre: Skipped 1 previous similar message [ 5152.788081] Lustre: server umount lustre-MDT0000 complete [ 5155.810381] LustreError: 152565:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5155.842430] LustreError: 152565:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5158.546050] LustreError: 152545:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784191992 with bad export cookie 7711197796847006868 [ 5158.547912] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5158.555558] LustreError: 152545:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5158.997573] Lustre: server umount lustre-MDT0001 complete [ 5172.496908] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing unload_modules_local [ 5176.094227] Key type lgssc unregistered [ 5176.535833] LNet: 156652:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5176.545539] LNetError: 156652:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5176.564380] LNet: Removed LNI 192.168.201.108@tcp [ 5177.577539] Key type .llcrypt unregistered [ 5177.578871] Key type ._llcrypt unregistered [ 5257.175650] Key type ._llcrypt registered [ 5257.179591] Key type .llcrypt registered [ 5257.281264] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 5258.275500] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5258.321879] alg: No test for adler32 (adler32-zlib) [ 5259.643198] Lustre: Lustre: Build Version: 2.17.54_162_g9958c38 [ 5259.902812] LNet: Added LNI 192.168.201.108@tcp [8/256/0/180] [ 5261.592116] Key type lgssc registered [ 5263.005172] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5271.847313] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5271.879482] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5273.438510] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5279.092113] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 5283.809307] LustreError: 159144:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5283.835814] LustreError: 159144:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5288.929684] LustreError: 159145:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5289.442732] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5289.831766] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5294.739632] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 5302.423154] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5308.884502] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5316.294375] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5316.613734] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5318.748138] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 5323.129578] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 5330.801752] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5334.144649] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5344.742456] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5344.747405] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5344.761165] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5349.345283] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5349.354752] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5349.369440] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5350.853321] Lustre: server umount lustre-OST0000 complete [ 5354.976498] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5354.988966] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5354.993559] Lustre: Skipped 2 previous similar messages [ 5360.097107] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5360.106190] Lustre: Skipped 1 previous similar message [ 5365.218117] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5365.224653] Lustre: Skipped 1 previous similar message [ 5369.313266] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5369.551607] Lustre: server umount lustre-MDT0000 complete [ 5370.362967] LustreError: 159144:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5370.387248] LustreError: 159144:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5373.814645] LustreError: 159126:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784192207 with bad export cookie 12103724658413965834 [ 5373.835237] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5374.333597] Lustre: server umount lustre-MDT0001 complete [ 5387.901410] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing unload_modules_local [ 5390.606710] Key type lgssc unregistered [ 5390.865274] LNet: 162709:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5390.879053] LNetError: 162709:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5391.922446] LNet: Removed LNI 192.168.201.108@tcp [ 5392.905799] Key type .llcrypt unregistered [ 5392.908390] Key type ._llcrypt unregistered [ 5419.255633] Key type ._llcrypt registered [ 5419.258772] Key type .llcrypt registered [ 5419.400692] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 5421.087727] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5421.100551] alg: No test for adler32 (adler32-zlib) [ 5422.241982] Lustre: Lustre: Build Version: 2.17.54_162_g9958c38 [ 5422.634503] LNet: Added LNI 192.168.201.108@tcp [8/256/0/180] [ 5424.399534] Key type lgssc registered [ 5425.731320] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5441.149294] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 04:57:54 (1784192274) [ 5457.696674] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_hostid [ 5467.407803] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 5505.442686] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5505.503643] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5506.816094] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5506.842618] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5506.923352] Lustre: lustre-MDT0000: new disk, initializing [ 5506.998956] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5507.019937] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5511.531760] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 5522.872117] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5523.066920] Lustre: 166974:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5523.158536] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5523.161693] Lustre: Skipped 1 previous similar message [ 5523.241640] Lustre: lustre-MDT0001: new disk, initializing [ 5523.358265] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5523.418245] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5523.446166] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5529.436311] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 5540.138770] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5547.424071] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5554.412047] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5554.664160] Lustre: lustre-OST0000: new disk, initializing [ 5554.668060] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5554.672804] Lustre: 168943:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5554.759033] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5560.708798] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 5564.997631] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5565.015975] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5565.161197] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5570.374029] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5580.770240] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5580.773927] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5580.781444] LustreError: Skipped 1 previous similar message [ 5580.782534] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5580.799080] Lustre: Skipped 1 previous similar message [ 5585.669188] Lustre: server umount lustre-OST0000 complete [ 5589.985761] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5589.993217] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5590.018378] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5590.034367] Lustre: Skipped 1 previous similar message [ 5591.009678] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5591.027411] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5591.029918] Lustre: Skipped 1 previous similar message [ 5595.547280] Lustre: server umount lustre-MDT0000 complete [ 5599.138566] LustreError: 166967:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784192433 with bad export cookie 14867029753968045771 [ 5599.142659] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5599.148298] LustreError: 166967:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5599.409869] Lustre: server umount lustre-MDT0001 complete [ 5608.126229] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing unload_modules_local [ 5611.021212] Key type lgssc unregistered [ 5611.381507] LNet: 170851:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5611.390277] LNetError: 170851:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5611.411268] LNet: Removed LNI 192.168.201.108@tcp [ 5612.310413] Key type .llcrypt unregistered [ 5612.314498] Key type ._llcrypt unregistered [ 5637.071355] Key type ._llcrypt registered [ 5637.073720] Key type .llcrypt registered [ 5637.211148] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 5638.554785] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5638.566616] alg: No test for adler32 (adler32-zlib) [ 5639.685238] Lustre: Lustre: Build Version: 2.17.54_162_g9958c38 [ 5639.945526] LNet: Added LNI 192.168.201.108@tcp [8/256/0/180] [ 5641.631297] Key type lgssc registered [ 5642.980437] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5652.187781] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5652.245994] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5654.111486] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5660.191655] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 5664.236938] LustreError: 172280:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5664.284457] LustreError: 172280:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5668.819082] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5669.056081] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5674.047328] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 5683.041973] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5690.522135] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5699.093539] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5699.663505] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5707.225778] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 5715.983462] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5724.279834] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5733.215264] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5733.250567] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5733.275601] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5733.291115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.311191] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5734.449370] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5734.471174] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5734.487740] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5734.492745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.506913] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5735.690337] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5735.707355] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5735.715994] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5735.721854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.728919] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5736.708350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.675913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.662396] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5738.666820] Lustre: Skipped 2 previous similar messages [ 5738.683174] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5738.692821] Lustre: Skipped 2 previous similar messages [ 5738.696845] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5738.701896] Lustre: Skipped 2 previous similar messages [ 5738.705810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.713529] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5738.716759] Lustre: Skipped 2 previous similar messages [ 5739.578169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.433730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.388257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.327496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.575074] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5743.580391] Lustre: Skipped 4 previous similar messages [ 5743.596124] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5743.608935] Lustre: Skipped 4 previous similar messages [ 5743.614137] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5743.620220] Lustre: Skipped 4 previous similar messages [ 5743.622844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.633353] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5743.643652] Lustre: Skipped 4 previous similar messages [ 5744.678204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.619547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.613632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.610129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.514442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.632789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.611419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.438621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.430353] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5752.437950] Lustre: Skipped 8 previous similar messages [ 5752.467877] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5752.480731] Lustre: Skipped 8 previous similar messages [ 5752.484022] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5752.489737] Lustre: Skipped 8 previous similar messages [ 5752.495548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.507815] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5752.510663] Lustre: Skipped 8 previous similar messages [ 5753.385721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.465418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.261321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.186236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.156813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.285606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.397618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.295409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.330783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.284263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.372732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.418769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.438200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.629309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.702606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.700375] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5768.706833] Lustre: Skipped 15 previous similar messages [ 5768.736811] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5768.751375] Lustre: Skipped 15 previous similar messages [ 5768.757373] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5768.762894] Lustre: Skipped 15 previous similar messages [ 5768.769721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.783683] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5768.786506] Lustre: Skipped 15 previous similar messages [ 5769.797608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.764374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.794406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.995237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.949821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.042700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.255872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.306278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.172361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.221204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.210632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.121074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.178682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.259406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.238122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.294899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.282682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.275575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.289879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.376055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.261360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.311424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.253258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.485235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.500501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.551850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.547697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.628141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.806654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.765884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.710479] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5800.716461] Lustre: Skipped 30 previous similar messages [ 5800.724628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.503129] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5801.512979] Lustre: Skipped 31 previous similar messages [ 5801.515531] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5801.519195] Lustre: Skipped 31 previous similar messages [ 5801.521981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.540475] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5801.545512] Lustre: Skipped 31 previous similar messages [ 5802.506936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.399219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.312878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.220438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.470647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.536899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.406826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.393602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.477847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.414528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.416673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.429286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.588543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.511476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.607824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.670318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.822175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.831627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.960828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.128898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.252067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.329803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.513424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.545451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.670153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.744656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.819124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.734172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.694471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.629938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.823892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.839654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.998336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.976710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.882170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.830381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.454295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.578839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.485691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.533509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.495039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.373391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.257030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.330682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.020960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.982657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.893895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.223889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.474330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.647244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.188604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.554962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.974728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.149659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.394684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.286873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.269601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.996746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.967096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.235276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.487096] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5865.494074] Lustre: Skipped 61 previous similar messages [ 5865.523540] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5865.533128] Lustre: Skipped 60 previous similar messages [ 5865.537296] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5865.540602] Lustre: Skipped 60 previous similar messages [ 5865.544666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.553241] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5865.559483] Lustre: Skipped 60 previous similar messages [ 5866.818516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.065531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.448544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.593960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.671620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.088587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.566786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.701849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.975680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.790489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.793529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.772772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.743662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.677815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.764266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.700387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.837785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.803855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.930343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.137097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.266943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.275311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.385763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.541653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.461087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.560540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.669157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.765354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.785304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.034852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.404464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.774105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.073720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.043794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.168152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.165228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.268631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.373749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.526308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.480509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.954607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.125690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.597274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.492976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.713859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.892381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.175923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.356488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.272428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.403439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.306712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.545680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.578983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.626979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.995807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.984665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.103771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.245439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.165036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.211414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.811377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.881849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.104907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.973509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.256254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.459159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.584552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.022469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.192306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.365361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.587328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.810320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.065898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.244688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.357883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.118800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.068791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.230274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.210767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.252152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.046564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.008773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.992327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.081364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.107694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.118556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.007266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.150697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.914608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.815189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.684416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.765152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.730431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.019989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.835874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.909277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.679738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.534869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.228047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.355885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.615871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.891831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.227968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.799168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.805571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.248933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.466844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.449195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.668780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.305570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.622385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.969978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.147753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.150296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.372569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.693884] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5994.699163] Lustre: Skipped 115 previous similar messages [ 5994.723449] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5994.740615] Lustre: Skipped 115 previous similar messages [ 5994.752660] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5994.755765] Lustre: Skipped 115 previous similar messages [ 5994.769909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.788281] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5994.791010] Lustre: Skipped 115 previous similar messages [ 5995.949056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.169116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.836822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.876547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.503794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.611969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.856963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.866376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.913066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.058419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.187834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.358842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.410080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.482778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.522765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.604112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.897782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.103994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.200800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.638777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.811973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.795912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.883618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.976766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.259757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.288886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.348124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.370299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.338447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.391243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.272132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.195513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.192234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.385948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.439114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.396085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.713626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.888505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.881382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.749670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.714399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.838070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.833927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.863681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.934576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.162191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.096037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.253791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.375308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.303607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.475559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.434089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.541518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.576077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.444772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.526340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.609724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.569585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.664435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.731957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.891376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.832455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.740253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.525249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.468675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.345712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.534293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.749558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.170706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.504904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.832362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.638134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.720766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.706209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.927810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.950541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.115130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.507557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.580291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.682362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.748842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.736458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.931986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.756918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.855189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.732083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.812961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.763631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.797244] LustreError: 172299:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6089.806125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.809738] LustreError: 172299:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6090.756834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.720445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.719454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.737688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.707373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.657454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.649654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.653656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.726465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.610513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.544918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.679793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.689672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.666197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.524592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.678638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.698550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.747557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.767090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.736491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.652070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.714208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.695790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.702942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.809318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.966233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.084559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.221837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.277608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.430658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.610134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.766993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.669881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.788363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.783236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.816380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.743815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.741898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.598250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.660443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.589477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.537852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.390400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.367373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.278742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.434230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.465182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.463318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.776134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.973022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.203751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.424060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.285578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.665313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.597973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.983351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.108702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.280846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.489974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.678838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.649738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.708262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.516270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.664612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.741602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.984129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.079931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.509887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.469327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.880056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.258654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.668897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.019907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.237295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.611404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.185585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.412197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.553659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.908668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.247578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.266602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.273396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.559665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.104276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.712310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.049586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.209337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.595504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.648138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.787542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.811782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.852460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.856902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.958271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.800738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.870144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.756128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.862702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.842444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.817572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.971812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.086938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.019912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.264896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.250123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.183082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.144073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.434527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.521943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.769388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.876442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.878204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.952869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.054962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.147099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.117300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.126433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.970767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.874516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.565192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.490741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.406395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.186479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.079590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.077444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.101361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.113988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.846645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.816167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.762764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.759759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.741611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.674716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.493432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.354715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.283173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.106582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.928582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.841976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.636528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.420348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.305051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.243881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.256537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.241816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.150573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.079758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.981965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.210880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.335133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.420829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.515572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.564579] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6251.569233] Lustre: Skipped 241 previous similar messages [ 6251.596555] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6251.605435] Lustre: Skipped 241 previous similar messages [ 6251.611849] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6251.624457] Lustre: Skipped 241 previous similar messages [ 6251.628972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.658033] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6251.664466] Lustre: Skipped 241 previous similar messages [ 6252.775242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.989440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.066455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.880209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.872942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.728810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.738070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.712285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.642200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.556470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.474034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.401156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.458795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.559561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.598536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.626269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.616740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.744451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.697823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.632150] LustreError: 172299:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6271.654505] LustreError: 172299:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6271.674200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.826039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.761723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.705772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.818751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.792079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.799350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.710889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.643754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.415609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.398315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.129200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.308706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.427313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.554366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.643823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.669398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.940564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.992498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.093601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.907913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.224324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.304177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.454567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.853215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.850562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.955768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.878389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.814190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.839425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.795375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.750883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.534708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.498461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.489990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.317225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.219210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.837455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.611373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.199501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.966897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.850977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.049451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.084941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.492785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.921766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.704554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.687453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.680368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.893261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.444697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.528817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.050975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.274516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.040889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.061396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.521425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.168839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.826038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.462887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.993401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.808967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6343.434893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.636818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.059108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.411180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6348.860568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.825768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6351.142470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.450951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.934407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6354.878981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.239699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.678809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.848701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.917936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6361.334196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6362.856227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.105771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.978359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.966429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.938437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.729882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.538809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.460644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.245114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.123904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.058115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6373.012273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6373.828117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.700285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.602360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.510512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6377.146255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6377.985631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.740464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.687537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.523840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.788199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6382.584891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.250850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6384.284887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6385.106289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6386.195564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.500758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6388.367916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6389.668941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6390.455447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.331877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6392.107327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6392.877960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.779645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6394.886739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6395.843415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.001609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.343167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6399.213934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.065814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.986437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.031316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.936255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6403.904608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6404.865118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.784530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6406.759456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6407.786153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6408.645487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6409.672259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.515575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.644905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6412.707656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.774163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6414.884414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.515638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.771454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.900584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6418.609286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.292900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.973368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.924543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6422.159133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6423.081247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6423.960069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.608988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.551526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6426.223179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6427.023205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6427.733859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6428.416287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6429.037127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6429.700125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.206982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.923708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.545879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6432.171405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6432.754574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6433.251657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6433.748249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6434.259350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6434.775963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.313319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.895218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6436.396124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6437.012805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6437.572314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.083735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.480434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.904795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6439.280674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6439.785709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6440.227299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6440.623506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.029225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.409772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.773558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.143698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.581264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6443.014684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6443.428840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6443.854851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6444.256286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6444.669969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.082078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.483485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.906186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6446.371082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6446.809607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6447.237606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6447.668974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6448.031994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6448.382288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6448.745397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6449.138034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6449.603288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6450.042379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6450.406620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6450.805120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.176240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.517214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.862099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6452.195384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6452.545931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6452.921640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6453.308074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6453.667931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6454.026961] LustreError: 172299:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6454.029719] LustreError: 172299:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6454.031989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6454.366477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6454.731289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.114644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.471097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.807721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6456.153917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6456.506541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6456.843477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6457.184079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6457.521410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6457.869111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6458.214940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6458.555835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6458.881334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6459.238935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6459.587989] LustreError: 172299:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6459.590527] LustreError: 172299:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6459.595937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6459.934268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6460.257923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6460.737510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6462.000899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6463.464098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6464.789296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6466.106222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6467.148900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6468.328319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6469.341253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6470.569901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6472.085464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6473.247826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6474.853049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6476.346508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6477.568097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6478.989351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6480.234222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6481.465165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6482.659723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6483.604214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6484.357316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6485.163747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6486.155773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6487.187802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6488.246744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6489.190824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6490.161305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6491.101577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6492.071452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6493.013253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6494.087074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6494.968533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6495.725471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6496.571634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6497.432822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6498.345274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6499.287975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6500.340469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6501.325849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6502.317764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6503.172580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6504.443059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6505.474506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6506.434538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6507.326589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6508.375667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6509.625659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6510.623338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6511.753457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6512.975321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6514.300976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6515.259659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6516.467716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6517.586200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6519.178383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6520.604386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6522.007939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6523.165422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6524.607807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6525.496326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6527.174743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6528.498253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6529.319453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6530.547589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6531.753906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6532.887612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6534.075033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6535.183375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6536.556143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6537.682892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6538.507683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6539.471251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6540.717912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6541.809225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6543.132922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6544.204533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6545.464061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6546.712546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6547.821585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6548.721452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6549.629682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6550.638654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6551.553990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6552.847051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6554.001839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6555.091709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6556.337739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6557.583588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6559.220243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6560.428133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6561.643954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6562.737457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6563.919572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6565.144651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6566.295388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6567.461929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6568.384714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6569.195590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6570.508407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6571.697785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6573.160775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6574.315470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6575.226979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6576.337470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6577.208118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6578.263771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6579.554340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6580.547441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6581.488966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6582.484339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6583.430662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6584.634331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6585.766287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6586.646506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6587.961737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6588.980087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6590.111925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6591.039200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6591.801657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6592.755794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6593.679212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6594.752100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6595.897333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6597.179732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6598.404797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6599.735325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6600.972172] LustreError: 172299:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6600.984063] LustreError: 172299:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6601.006186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6602.355469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6604.080489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6605.251445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6606.070229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6607.112210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6608.027193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6609.145324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6610.080455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6611.165890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6612.298245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6613.245260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6614.463671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6615.495540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6616.866258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6618.075777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6619.170860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6620.241746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6621.262172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6622.289720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6623.342978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6624.357167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6625.421663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6626.425733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6627.530944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6628.708611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6629.805466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6630.768193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6631.834408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6632.712765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6633.733883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6634.939813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6636.058826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6637.279734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6638.135083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6639.256837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6640.140905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6641.239802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6642.059993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6643.163399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6644.327487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6645.474114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6646.401355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6647.416583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6648.084316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6649.097078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6649.824400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6650.748223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6651.974969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6653.328873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6654.359463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6655.438870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6656.453774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6657.703972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6659.070445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6660.440194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6661.513622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6662.682528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6663.805450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6665.079891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6666.780854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6667.912078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6668.956684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6670.012695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6671.192292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6672.454202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6673.528300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6674.284640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6675.306456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6676.222969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6677.326961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6678.557350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6679.952251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6681.155946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6682.240438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6683.658420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6684.795895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6685.827537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6686.668555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6687.904466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6688.844973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6690.040971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6691.048363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6692.228418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6693.222580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6694.353150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6695.288763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6696.222706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6697.068552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6698.146300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6699.101919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6700.303746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6701.395676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6702.572239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6703.769911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6704.798732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6705.879534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6706.974123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6708.073868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6709.050748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6710.095579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6710.972493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6711.912069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6713.102227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6714.121641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6714.986079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6715.923154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6716.662416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6717.581884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6718.492125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6719.276638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6720.099452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6720.976619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6721.976850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6722.897661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6723.908561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6724.738806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6725.832099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6726.958688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6727.916480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6728.982942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6729.950259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6730.928245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6731.904176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6732.927803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6734.239401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6735.447747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6736.623659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6737.627306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6738.654980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6739.923362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6741.168705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6742.314513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6743.323748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6744.413379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6745.592058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6746.498628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6747.410939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6748.333127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6749.576036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6750.655240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6751.715090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6752.515521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6753.494482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6755.067789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7202.008212] LustreError: 174102:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 7202.012363] LustreError: 172299:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7903.712940] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7903.721127] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7903.738543] Lustre: Skipped 513 previous similar messages [ 7903.754829] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7905.256456] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7905.272827] Lustre: Skipped 1 previous similar message [ 7910.058778] Lustre: server umount lustre-OST0000 complete [ 7915.493331] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7920.609848] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7920.615518] Lustre: Skipped 1 previous similar message [ 7925.728964] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7925.754578] Lustre: Skipped 2 previous similar messages [ 7929.311584] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7929.840240] Lustre: server umount lustre-MDT0000 complete [ 7931.884728] LustreError: 172280:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7931.901809] LustreError: 172280:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7933.751595] LustreError: 174124:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784194767 with bad export cookie 2142390778770869172 [ 7933.768322] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7933.775834] LustreError: 174124:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7934.132443] Lustre: server umount lustre-MDT0001 complete [ 7945.538198] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing unload_modules_local [ 7948.246942] Key type lgssc unregistered [ 7948.596848] LNet: 227578:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7948.612525] LNetError: 227578:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7948.658830] LNet: Removed LNI 192.168.201.108@tcp [ 7950.101381] Key type .llcrypt unregistered [ 7950.103103] Key type ._llcrypt unregistered [ 7979.201383] Key type ._llcrypt registered [ 7979.203469] Key type .llcrypt registered [ 7979.336765] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 7980.353667] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7980.412150] alg: No test for adler32 (adler32-zlib) [ 7981.522582] Lustre: Lustre: Build Version: 2.17.54_162_g9958c38 [ 7981.829505] LNet: Added LNI 192.168.201.108@tcp [8/256/0/180] [ 7983.543262] Key type lgssc registered [ 7984.581937] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7994.346236] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7994.379059] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7996.288222] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8001.410427] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 8006.626948] LustreError: 228996:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8006.644568] LustreError: 228996:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 8010.726650] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8011.123234] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8016.572833] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 8025.165619] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8032.237683] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8040.710750] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8040.933650] LustreError: 230816:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8041.206594] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8046.587791] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 8048.555248] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 8056.676926] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8061.475415] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 8063.321052] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8063.329680] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 8063.333851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8063.347054] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 8063.347436] LustreError: 229016:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 8068.066433] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8068.072966] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8068.089289] Lustre: Skipped 1 previous similar message [ 8068.099968] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8068.103600] Lustre: Skipped 1 previous similar message [ 8068.718180] Lustre: lustre-OST0000: Not available for connect from 192.168.201.8@tcp (stopping) [ 8072.135255] Lustre: server umount lustre-OST0000 complete [ 8083.439089] LustreError: 229016:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 8108.147612] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8108.563465] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8114.664303] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 8124.075527] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8125.550173] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 8131.690590] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffffa0f983c6e000.ost_server_uuid 50 [ 8133.159148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8133.200649] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8133.201434] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 8133.223200] Lustre: Skipped 1 previous similar message [ 8133.226599] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffffa0f983c6e000.ost_server_uuid in IDLE state after 0 sec [ 8138.219864] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8138.226900] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8138.239307] Lustre: Skipped 1 previous similar message [ 8138.261656] Lustre: Skipped 1 previous similar message [ 8143.336959] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8143.352216] Lustre: Skipped 1 previous similar message [ 8143.869724] Lustre: server umount lustre-OST0000 complete [ 8153.570159] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8153.574811] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8153.579387] Lustre: Skipped 1 previous similar message [ 8153.617626] Lustre: Skipped 1 previous similar message [ 8163.816812] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8163.828883] Lustre: Skipped 3 previous similar messages [ 8164.319619] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8164.642237] Lustre: server umount lustre-MDT0000 complete [ 8167.920767] LustreError: 231321:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8167.963810] LustreError: 231321:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8170.563625] LustreError: 230839:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784195004 with bad export cookie 14369695323337011381 [ 8170.600803] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8170.608151] LustreError: 230839:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8171.236853] Lustre: server umount lustre-MDT0001 complete [ 8183.017450] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing unload_modules_local [ 8186.293528] Key type lgssc unregistered [ 8186.642794] LNet: 234208:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8186.649628] LNetError: 234208:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8186.687774] LNet: Removed LNI 192.168.201.108@tcp [ 8187.730353] Key type .llcrypt unregistered [ 8187.739185] Key type ._llcrypt unregistered [ 8224.854309] Key type ._llcrypt registered [ 8224.855928] Key type .llcrypt registered [ 8224.997337] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 8226.166970] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8226.306176] alg: No test for adler32 (adler32-zlib) [ 8227.358276] Lustre: Lustre: Build Version: 2.17.54_162_g9958c38 [ 8227.556940] LNet: Added LNI 192.168.201.108@tcp [8/256/0/180] [ 8229.247940] Key type lgssc registered [ 8231.034189] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8242.439492] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8242.466099] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8244.076878] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8249.023178] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 8254.456873] LustreError: 235934:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8254.487290] LustreError: 235934:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 8259.580227] LustreError: 235935:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8260.582105] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8261.313153] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8267.669177] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 8271.852212] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 8281.024829] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8281.597609] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8289.792071] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 8290.527375] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 8300.284104] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8300.589840] Lustre: lustre-OST0001: new disk, initializing [ 8300.597910] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8300.610641] Lustre: 238245:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8300.746267] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8307.418825] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8307.436843] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8307.542103] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8309.106848] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 8322.678643] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8330.482457] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 8348.131745] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8348.149100] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8348.171804] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8348.652818] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8348.663453] Lustre: Skipped 1 previous similar message [ 8348.674251] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8348.681861] Lustre: Skipped 2 previous similar messages [ 8352.203234] Lustre: server umount lustre-MDT0000 complete [ 8353.761993] LustreError: 235934:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8353.790654] LustreError: 235934:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 8356.888932] LustreError: 235916:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784195191 with bad export cookie 6952863456771036136 [ 8356.904547] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8356.914065] LustreError: 235916:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8357.485752] Lustre: server umount lustre-MDT0001 complete [ 8372.480783] Lustre: server umount lustre-OST0000 complete [ 8375.073124] Lustre: 235199:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1784195193/real 1784195193] req@ffff95489de49f80 x1870864120385792/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1784195209 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8375.111899] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8376.796922] Lustre: server umount lustre-OST0001 complete [ 8403.200616] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 8417.335441] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8418.096960] LustreError: 241354:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8418.255263] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8423.392176] LustreError: 241355:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8424.693641] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 8433.635682] LustreError: 241355:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8433.646163] LustreError: 241355:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8435.822150] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8436.470377] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8443.230753] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 8446.891948] Lustre: 242496:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8455.091115] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8455.787156] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8460.911528] LustreError: 242850:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8460.943444] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:129) [ 8462.989341] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 8463.788352] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 8474.117969] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8479.767404] Lustre: mylustre-OST0001: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:33) [ 8482.334894] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 8489.881479] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8494.127886] Lustre: 244370:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8507.877197] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8507.890153] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8507.928517] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8510.434305] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8510.447412] Lustre: Skipped 3 previous similar messages [ 8513.907606] Lustre: server umount mylustre-MDT0000 complete [ 8515.558991] LustreError: 241355:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8515.571771] LustreError: 241355:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 9 previous similar messages [ 8517.986824] LustreError: 241330:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784195352 with bad export cookie 6952863456771038999 [ 8518.003431] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8518.007168] LustreError: 241330:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8518.424215] Lustre: server umount mylustre-MDT0001 complete [ 8532.284668] Lustre: server umount mylustre-OST0000 complete [ 8546.300319] Lustre: server umount mylustre-OST0001 complete [ 8569.295051] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 8580.912133] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8581.731417] LustreError: 246679:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8581.974442] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8581.977456] Lustre: Skipped 1 previous similar message [ 8589.110981] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 8599.479285] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8604.697951] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 8607.702525] Lustre: 247821:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8614.188548] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8623.439220] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 8623.945229] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 8626.015751] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 8635.400650] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8641.049921] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:65) [ 8643.924667] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 8653.029610] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8657.984425] Lustre: 249693:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8671.722420] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8671.728694] Lustre: tfs-MDT0000-lwp-OST0000: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8671.736943] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 8671.753901] Lustre: Skipped 6 previous similar messages [ 8673.474268] Lustre: server umount tfs-MDT0000 complete [ 8677.712300] LustreError: 246655:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784195511 with bad export cookie 6952863456771041176 [ 8677.722408] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8677.725245] LustreError: 246655:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8678.188662] Lustre: server umount tfs-MDT0001 complete [ 8693.371524] Lustre: server umount tfs-OST0000 complete [ 8707.167170] Lustre: server umount tfs-OST0001 complete [ 8730.830901] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 8742.225209] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8742.722531] LustreError: 251946:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8742.734964] LustreError: 251946:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 14 previous similar messages [ 8742.785155] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8742.791053] Lustre: Skipped 3 previous similar messages [ 8747.897679] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 8757.767845] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8763.382485] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 8766.811614] Lustre: 253088:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8774.890127] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8782.328354] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 8783.681323] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 8787.763195] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 8792.418496] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8798.213341] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:97) [ 8799.886540] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 8809.178497] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8813.487396] Lustre: 254963:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8821.318788] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 05:54:14 (1784195654) [ 8839.139285] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8839.141750] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8839.155507] Lustre: Skipped 1 previous similar message [ 8839.168880] Lustre: Skipped 4 previous similar messages [ 8844.412686] Lustre: server umount lustre-MDT0000 complete [ 8848.698073] LustreError: 251922:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784195682 with bad export cookie 6952863456771043094 [ 8848.709296] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8848.717300] LustreError: 251922:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8858.529882] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8858.532733] Lustre: Skipped 4 previous similar messages [ 8863.201397] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8863.504655] Lustre: server umount lustre-MDT0001 complete [ 8873.640754] Lustre: server umount lustre-OST0000 complete [ 8883.913584] Lustre: server umount lustre-OST0001 complete [ 8892.212802] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_hostid [ 8904.999502] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 8944.030473] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 8944.030473] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8944.030473] [ 8944.070905] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8944.381037] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8944.411403] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8944.498773] Lustre: lustre-MDT0000: new disk, initializing [ 8944.626290] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8944.629482] Lustre: Skipped 3 previous similar messages [ 8944.646086] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8949.557586] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 8961.943825] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 8961.943825] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8961.943825] [ 8961.965237] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8962.114650] Lustre: 258148:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8962.189610] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8962.198462] Lustre: Skipped 1 previous similar message [ 8962.333383] Lustre: lustre-MDT0001: new disk, initializing [ 8962.574890] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8962.598616] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8968.073817] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 8980.385967] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8980.956717] Lustre: lustre-OST0000: new disk, initializing [ 8980.960696] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8980.968581] Lustre: 259734:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8982.677159] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8982.693592] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8982.823876] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8988.300782] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 9000.319949] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9000.500196] Lustre: lustre-OST0001: new disk, initializing [ 9000.503990] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 9000.510721] Lustre: 260605:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9002.102285] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 9002.109645] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 9002.238029] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 9008.009909] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 9020.848503] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 05:57:34 (1784195854) [ 9035.373875] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 05:57:49 (1784195869) [ 9053.667858] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9053.688742] Lustre: Skipped 4 previous similar messages [ 9053.693637] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9056.779113] Lustre: server umount lustre-OST0000 complete [ 9058.809339] LustreError: 259830:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9058.838202] LustreError: 259830:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 17 previous similar messages [ 9075.168257] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9075.788952] Lustre: server umount lustre-MDT0000 complete [ 9079.835962] LustreError: 258141:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784195914 with bad export cookie 6952863456771044872 [ 9079.836834] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9079.847566] LustreError: 258141:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9086.310774] Lustre: server umount lustre-MDT0001 complete [ 9098.208917] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing unload_modules_local [ 9111.089976] Lustre: server umount lustre-OST0001 complete [ 9118.726567] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_hostid [ 9135.202244] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 9173.044897] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9173.266655] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9173.294680] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9173.356995] Lustre: lustre-MDT0000: new disk, initializing [ 9173.445053] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9178.443298] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 9188.898142] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9189.080179] Lustre: 265167:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9189.165395] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9189.169551] Lustre: Skipped 1 previous similar message [ 9189.276402] Lustre: lustre-MDT0001: new disk, initializing [ 9189.454511] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9189.478812] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9194.484475] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 9204.898845] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9210.720032] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9218.717571] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9219.163654] Lustre: lustre-OST0000: new disk, initializing [ 9219.166572] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9219.174184] Lustre: 267137:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9219.265272] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9219.270338] Lustre: Skipped 5 previous similar messages [ 9220.878856] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9220.886791] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9220.985850] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9226.537526] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 9238.067470] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9246.689055] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9246.696729] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9246.707850] Lustre: Skipped 5 previous similar messages [ 9246.717242] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9246.726890] Lustre: Skipped 12 previous similar messages [ 9252.137612] Lustre: server umount lustre-OST0000 complete [ 9262.494089] Lustre: server umount lustre-MDT0000 complete [ 9266.133094] LustreError: 265160:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784196100 with bad export cookie 6952863456771046342 [ 9266.135784] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9266.141862] LustreError: 265160:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9266.427950] Lustre: server umount lustre-MDT0001 complete [ 9280.450510] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing unload_modules_local [ 9284.371897] Key type lgssc unregistered [ 9284.659783] LNet: 268911:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9284.667526] LNetError: 268911:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9284.687466] LNet: Removed LNI 192.168.201.108@tcp [ 9285.744143] Key type .llcrypt unregistered [ 9285.747749] Key type ._llcrypt unregistered [ 9316.473381] Key type ._llcrypt registered [ 9316.476646] Key type .llcrypt registered [ 9316.643855] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 9318.018826] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9318.052332] alg: No test for adler32 (adler32-zlib) [ 9319.556408] Lustre: Lustre: Build Version: 2.17.54_162_g9958c38 [ 9319.923905] LNet: Added LNI 192.168.201.108@tcp [8/256/0/180] [ 9321.687146] Key type lgssc registered [ 9323.554090] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9332.667163] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9332.683047] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9334.246050] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9339.746926] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 9344.488113] LustreError: 270375:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9344.508553] LustreError: 270375:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 9349.409723] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9349.601451] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 9349.755526] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9354.994890] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 9363.333541] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9370.947945] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9379.531049] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9379.814875] LustreError: 272193:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9380.200754] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9385.513954] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 9388.796253] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 9398.535564] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9399.745357] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 9404.898107] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9404.901510] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9404.923603] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9410.033822] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9410.065989] Lustre: Skipped 1 previous similar message [ 9415.141128] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9415.148020] Lustre: Skipped 1 previous similar message [ 9416.159164] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9416.320255] Lustre: server umount lustre-OST0000 complete [ 9420.258394] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9420.260576] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9420.268905] Lustre: Skipped 2 previous similar messages [ 9420.271032] Lustre: Skipped 1 previous similar message [ 9425.843852] Lustre: server umount lustre-MDT0000 complete [ 9429.846770] LustreError: 272904:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784196264 with bad export cookie 14058343521236543880 [ 9429.849430] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9429.850126] LustreError: 272904:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9430.147995] Lustre: server umount lustre-MDT0001 complete [ 9442.812456] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing unload_modules_local [ 9445.873636] Key type lgssc unregistered [ 9446.306388] LNet: 273962:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9446.322146] LNetError: 273962:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9446.347092] LNet: Removed LNI 192.168.201.108@tcp [ 9447.576745] Key type .llcrypt unregistered [ 9447.580370] Key type ._llcrypt unregistered [ 9462.606299] Key type ._llcrypt registered [ 9462.608722] Key type .llcrypt registered [ 9462.784873] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 9464.192262] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9464.235140] alg: No test for adler32 (adler32-zlib) [ 9465.504322] Lustre: Lustre: Build Version: 2.17.54_162_g9958c38 [ 9465.748481] LNet: Added LNI 192.168.201.108@tcp [8/256/0/180] [ 9467.463195] Key type lgssc registered [ 9468.698038] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9478.765158] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9478.790642] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9494.368510] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 9499.487425] LustreError: 275136:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.108@tcp: failed processing log, type 4: rc = -110 [ 9526.175413] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9533.751547] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 9541.905724] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9544.105107] Lustre: server umount lustre-OST0000 complete [ 9562.169871] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing unload_modules_local [ 9564.966245] Key type lgssc unregistered [ 9565.331696] LNet: 276768:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9565.355312] LNetError: 276768:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9565.386775] LNet: Removed LNI 192.168.201.108@tcp [ 9566.227720] Key type .llcrypt unregistered [ 9566.233562] Key type ._llcrypt unregistered [ 9603.720610] Key type ._llcrypt registered [ 9603.722444] Key type .llcrypt registered [ 9603.801182] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 9604.938043] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9604.978620] alg: No test for adler32 (adler32-zlib) [ 9605.960142] Lustre: Lustre: Build Version: 2.17.54_162_g9958c38 [ 9606.165239] LNet: Added LNI 192.168.201.108@tcp [8/256/0/180] [ 9607.840811] Key type lgssc registered [ 9608.848895] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9623.990373] loop0: detected capacity change from 0 to 204800000 [ 9624.246601] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9624.261516] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9624.685875] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9625.984221] loop0: detected capacity change from 0 to 204800000 [ 9626.165444] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9626.178199] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9626.606237] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9627.647699] loop0: detected capacity change from 0 to 204800000 [ 9627.824629] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9627.839242] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9627.947811] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9629.043330] loop0: detected capacity change from 0 to 204800000 [ 9629.333643] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9629.346297] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9629.519597] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9630.656365] loop0: detected capacity change from 0 to 204800000 [ 9630.676350] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9634.923914] loop0: detected capacity change from 0 to 204800000 [ 9634.946195] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9639.318745] loop0: detected capacity change from 0 to 204800000 [ 9639.354197] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9643.954016] loop0: detected capacity change from 0 to 204800000 [ 9643.994400] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9647.876589] loop0: detected capacity change from 0 to 204800000 [ 9647.962931] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9648.030781] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9648.070507] Lustre: 280177:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9648.077067] Lustre: 280177:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 9648.084117] Lustre: 280177:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 9648.090821] Lustre: 280177:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9648.098679] Lustre: 280177:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9648.108197] Lustre: 280177:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 9648.116596] Lustre: 280177:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 9648.125187] Lustre: 280177:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9648.136219] Lustre: 280177:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 9648.146703] Lustre: 280177:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 9648.167878] Lustre: 280177:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 9648.186481] Lustre: 280177:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 9648.196838] Lustre: 280177:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 9648.210933] Lustre: 280177:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 9653.993132] Lustre: server umount MGS complete [ 9655.034607] loop0: detected capacity change from 0 to 204800000 [ 9655.100751] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9655.797696] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9655.859514] Lustre: lustre-MDT0000: Aborting client recovery [ 9656.852286] loop1: detected capacity change from 0 to 204800000 [ 9656.984525] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9657.083402] Lustre: 280608:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9657.094343] Lustre: 280608:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 9657.105052] Lustre: 280608:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9657.112687] Lustre: 280608:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9657.121287] Lustre: 280608:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 9657.126974] Lustre: 280608:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9657.139978] Lustre: 280608:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 9657.160873] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9657.292210] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9657.345678] Lustre: lustre-MDT0001: Aborting client recovery [ 9658.285558] loop2: detected capacity change from 0 to 204800000 [ 9658.474212] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9658.539888] Lustre: 280687:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9658.549427] Lustre: 280687:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9658.558345] Lustre: 280687:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9658.564802] Lustre: 280687:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9658.570931] Lustre: 280687:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9658.577851] Lustre: 280687:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9658.584372] Lustre: 280687:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9658.712872] Lustre: Found index 0 for lustre-OST0000, updating log [ 9658.747244] Lustre: 280706:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9658.821950] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9658.847477] Lustre: lustre-OST0000: Aborting recovery [ 9659.905571] loop3: detected capacity change from 0 to 204800000 [ 9660.048566] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9660.114844] Lustre: 280772:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9660.130360] Lustre: 280772:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9660.140875] Lustre: 280772:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9660.158625] Lustre: 280772:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9660.171610] Lustre: 280772:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9660.199731] Lustre: 280772:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9660.224708] Lustre: 280772:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9660.252501] Lustre: Found index 1 for lustre-OST0001, updating log [ 9660.296862] Lustre: 280777:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9660.359864] Lustre: lustre-OST0001: Aborting recovery [ 9665.458099] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9665.465512] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9665.494810] LustreError: 280552:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 9665.540133] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 9665.540785] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 9667.497863] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 9667.505988] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 9668.784513] LustreError: 280518:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 11965us: rc = -2 [ 9669.294808] LustreError: 280517:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 4594us: rc = -2 [ 9669.317404] LustreError: 280517:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 19 previous similar messages [ 9682.915617] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9682.927440] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9682.932106] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9682.943258] Lustre: Skipped 3 previous similar messages [ 9682.955079] Lustre: Skipped 3 previous similar messages [ 9688.040182] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9688.054797] Lustre: Skipped 3 previous similar messages [ 9688.540068] Lustre: server umount lustre-MDT0000 complete [ 9689.646227] LustreError: 280502:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784196523 with bad export cookie 4387963197477887016 [ 9689.662481] LustreError: 280502:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9689.664462] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9693.152182] LustreError: 280515:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9693.152684] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9693.173919] LustreError: 280515:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 9693.196365] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9693.202696] Lustre: Skipped 1 previous similar message [ 9694.176165] LustreError: 280517:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9694.198994] LustreError: 280517:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9696.105436] Lustre: server umount lustre-MDT0001 complete [ 9703.164881] Lustre: server umount lustre-OST0000 complete [ 9718.241128] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9718.762562] Lustre: server umount lustre-OST0001 complete [ 9727.559269] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 06:09:20 (1784196560) [ 9759.663994] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_hostid [ 9770.200133] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [ 9813.750066] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9813.976737] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9814.006195] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9814.121924] Lustre: lustre-MDT0000: new disk, initializing [ 9814.265275] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9814.273948] Lustre: Skipped 1 previous similar message [ 9814.292298] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9819.086799] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 9832.908164] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9833.145462] Lustre: 284842:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9833.278106] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9833.283121] Lustre: Skipped 1 previous similar message [ 9833.385232] Lustre: lustre-MDT0001: new disk, initializing [ 9833.533802] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9833.571892] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9833.584353] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9837.325796] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 9848.856422] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9856.224687] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9863.622454] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9863.936926] Lustre: lustre-OST0000: new disk, initializing [ 9863.940326] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9863.946723] Lustre: 286811:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9864.030862] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9871.347652] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 9873.011745] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9873.031720] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9873.149949] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9881.531401] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9887.795587] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9897.229366] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9903.158192] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9945.063310] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9945.064972] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9945.085058] Lustre: Skipped 2 previous similar messages [ 9945.087547] Lustre: Skipped 2 previous similar messages [ 9945.097561] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9949.561624] Lustre: server umount lustre-OST0000 complete [ 9955.300589] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9955.304714] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9955.312415] Lustre: Skipped 1 previous similar message [ 9955.318393] Lustre: Skipped 1 previous similar message [ 9959.675603] Lustre: server umount lustre-MDT0000 complete [ 9960.416107] LustreError: 284854:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9960.432221] LustreError: 284854:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9963.751190] LustreError: 285637:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784196797 with bad export cookie 4387963197477889746 [ 9963.773162] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9963.774692] LustreError: 285637:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9964.143117] Lustre: server umount lustre-MDT0001 complete [ 9973.563295] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9978.583063] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [ 9983.404418] Lustre: server umount MGS complete [ 9994.153386] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9994.775292] LustreError: 290627:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9994.821908] LustreError: 290640:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9994.945808] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10000.357157] LustreError: 290628:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10000.483514] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [10009.573194] LustreError: 290627:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10009.590531] LustreError: 290627:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10010.284787] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10015.627712] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [10024.314483] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10030.607263] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10037.495158] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10037.936966] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10037.944328] Lustre: Skipped 1 previous similar message [10043.762057] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [10050.693584] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10080.230233] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10080.238286] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10080.251068] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10080.254879] Lustre: Skipped 1 previous similar message [10085.233276] Lustre: server umount lustre-OST0000 complete [10091.490206] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10091.508763] Lustre: Skipped 2 previous similar messages [10095.067110] Lustre: server umount lustre-MDT0000 complete [10096.608236] LustreError: 290627:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10096.622667] LustreError: 290627:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10098.730702] LustreError: 292473:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784196932 with bad export cookie 4387963197477891594 [10098.736878] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10098.741549] LustreError: 292473:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10098.968609] Lustre: server umount lustre-MDT0001 complete [10109.222621] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing unload_modules_local [10111.469326] Key type lgssc unregistered [10111.758311] LNet: 294573:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10111.768083] LNetError: 294573:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10111.801618] LNet: Removed LNI 192.168.201.108@tcp [10112.477603] Key type .llcrypt unregistered [10112.480100] Key type ._llcrypt unregistered [10143.766424] Key type ._llcrypt registered [10143.769823] Key type .llcrypt registered [10143.868318] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_hostid [10156.013212] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [10156.854176] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10156.975061] alg: No test for adler32 (adler32-zlib) [10157.961272] Lustre: Lustre: Build Version: 2.17.54_162_g9958c38 [10158.210765] LNet: Added LNI 192.168.201.108@tcp [8/256/0/180] [10159.855181] Key type lgssc registered [10160.842821] Lustre: Echo OBD driver; http://www.lustre.org/ [10190.477852] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10190.509070] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10191.752046] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10191.775869] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10191.841790] Lustre: lustre-MDT0000: new disk, initializing [10191.983992] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10191.995940] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10196.885333] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [10206.070035] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10206.169605] Lustre: 298507:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10206.210098] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10206.214220] Lustre: Skipped 1 previous similar message [10206.260962] Lustre: lustre-MDT0001: new disk, initializing [10206.340301] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10206.374372] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10206.386363] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10209.945591] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [10217.928560] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10223.416898] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10230.049902] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10230.395426] Lustre: lustre-OST0000: new disk, initializing [10230.405280] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10230.419474] Lustre: 300475:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10230.496018] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10236.475291] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [10237.024200] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10237.041437] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10237.207714] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10245.284937] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10251.519686] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10261.683790] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10270.940300] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10278.800726] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [10324.447801] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10324.452087] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10324.466467] LustreError: Skipped 1 previous similar message [10324.468373] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10324.480377] Lustre: Skipped 1 previous similar message [10324.501882] Lustre: Skipped 1 previous similar message [10326.568736] Lustre: server umount lustre-OST0000 complete [10334.690305] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10334.692565] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10334.711938] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10334.723243] Lustre: Skipped 1 previous similar message [10334.733291] Lustre: Skipped 1 previous similar message [10336.165567] Lustre: server umount lustre-MDT0000 complete [10339.763910] LustreError: 298501:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784197173 with bad export cookie 2856173083669835097 [10339.775931] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10340.181558] Lustre: server umount lustre-MDT0001 complete [10348.619023] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10353.254905] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [10357.402751] Lustre: server umount MGS complete [10367.676929] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10368.106870] LustreError: 304458:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10368.172896] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10371.577544] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [10373.604059] LustreError: 304446:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10373.629033] LustreError: 304446:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10378.171797] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10378.591916] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10382.242518] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [10388.659656] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10393.714260] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10400.001720] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10400.268626] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10406.166148] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [10413.071850] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10444.257684] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10444.270928] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10444.288627] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10448.444771] Lustre: server umount lustre-OST0000 complete [10455.520684] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10455.523223] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10455.530222] Lustre: Skipped 2 previous similar messages [10455.540931] Lustre: Skipped 3 previous similar messages [10458.562554] Lustre: server umount lustre-MDT0000 complete [10460.644452] LustreError: 305174:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10460.662384] LustreError: 305174:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10462.244660] LustreError: 304425:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784197296 with bad export cookie 2856173083669837036 [10462.245888] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10462.253859] LustreError: 304425:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [10462.572340] Lustre: server umount lustre-MDT0001 complete [10473.607924] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing unload_modules_local [10476.500774] Key type lgssc unregistered [10476.778821] LNet: 308389:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10476.783247] LNetError: 308389:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10476.807602] LNet: Removed LNI 192.168.201.108@tcp [10477.653429] Key type .llcrypt unregistered [10477.655957] Key type ._llcrypt unregistered [10496.734535] Key type ._llcrypt registered [10496.736807] Key type .llcrypt registered [10496.823708] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_hostid [10507.852820] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [10508.620484] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10508.753575] alg: No test for adler32 (adler32-zlib) [10509.900910] Lustre: Lustre: Build Version: 2.17.54_162_g9958c38 [10510.191153] LNet: Added LNI 192.168.201.108@tcp [8/256/0/180] [10511.895164] Key type lgssc registered [10512.884807] Lustre: Echo OBD driver; http://www.lustre.org/ [10533.788252] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [10546.328694] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10546.351414] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10547.607614] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10547.649691] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10547.727363] Lustre: lustre-MDT0000: new disk, initializing [10547.786493] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10547.812268] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10551.856182] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [10561.465604] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10561.682457] Lustre: lustre-OST0000: new disk, initializing [10561.686083] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10561.689736] Lustre: Skipped 1 previous similar message [10561.694896] Lustre: 312417:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10561.762986] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10566.643962] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [10571.303111] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [10571.311863] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [10571.340322] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [11211.232774] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11211.245724] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11215.759152] Lustre: server umount lustre-MDT0000 complete [11218.155783] LustreError: 313473:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784198052 with bad export cookie 9759221034247605318 [11218.161185] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11218.161298] LustreError: 313473:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11218.279768] Lustre: server umount lustre-OST0000 complete [11235.387989] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_hostid [11241.869752] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [11269.459194] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11269.580294] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11269.597625] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11269.648156] Lustre: lustre-MDT0000: new disk, initializing [11269.696825] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11269.704544] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11272.780847] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [11280.741332] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11280.798790] Lustre: 316741:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11280.834215] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11280.836793] Lustre: Skipped 1 previous similar message [11280.883612] Lustre: lustre-MDT0001: new disk, initializing [11280.948725] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11280.985779] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11280.993194] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11284.045265] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [11291.645990] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11295.602117] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11300.451420] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11300.638694] Lustre: lustre-OST0000: new disk, initializing [11300.641791] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11300.647166] Lustre: 318708:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11300.722911] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11304.731724] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [11310.124522] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11310.140241] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11310.272864] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11312.172302] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11315.929768] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11316.095549] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11319.491176] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11319.654556] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11325.923686] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11325.928608] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11325.929760] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11325.943480] LustreError: Skipped 1 previous similar message [11327.223374] Lustre: server umount lustre-OST0000 complete [11331.045969] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11331.054433] Lustre: Skipped 2 previous similar messages [11331.059662] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11331.065220] Lustre: Skipped 2 previous similar messages [11336.162859] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11336.169565] Lustre: Skipped 1 previous similar message [11341.282814] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11341.289744] Lustre: Skipped 1 previous similar message [11344.354578] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11344.511214] Lustre: server umount lustre-MDT0000 complete [11347.076157] LustreError: 318705:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784198181 with bad export cookie 9759221034250546137 [11347.080874] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11347.085291] LustreError: 318705:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11347.303057] Lustre: server umount lustre-MDT0001 complete [11349.380727] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [11350.652898] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 06:36:24 (1784198184) [11355.727367] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11356.117600] LustreError: 321020:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11356.133801] LustreError: 321020:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [11356.178355] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11359.193768] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [11361.256970] LustreError: 321021:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11365.097086] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11365.307831] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11367.526842] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [11371.398874] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11374.749603] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11378.143804] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11378.339766] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11381.217898] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [11384.591851] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11387.530787] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11387.617891] Lustre: lustre-OST0001: new disk, initializing [11387.620512] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [11387.623106] Lustre: 323850:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11387.802853] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [11387.806389] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [11387.825166] LustreError: 323887:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11388.834476] LustreError: 323887:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11388.838577] LustreError: 323887:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [11389.866743] LustreError: 323887:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11389.874028] LustreError: 323887:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [11390.239990] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [11391.905324] LustreError: 323887:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11391.908526] LustreError: 323887:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [11394.598492] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11394.687197] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11396.001541] LustreError: 323887:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11396.005051] LustreError: 323887:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [11396.895578] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffffa0f9b8a51000.ost_server_uuid 50 [11397.553943] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffffa0f9b8a51000.ost_server_uuid in FULL state after 0 sec [11399.739342] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffffa0f9b8a51000.ost_server_uuid 50 [11400.359674] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffffa0f9b8a51000.ost_server_uuid in FULL state after 0 sec [11401.194173] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [11413.634514] Lustre: lustre-OST0001: Not available for connect from 192.168.201.8@tcp (stopping) [11415.524552] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11415.534991] Lustre: Skipped 1 previous similar message [11427.295858] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11427.529386] Lustre: server umount lustre-OST0001 complete [11428.981390] LustreError: 325134:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.201.8@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11436.004285] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11436.005467] LustreError: 323892:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11436.006886] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11436.008502] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11436.008507] Lustre: Skipped 8 previous similar messages [11436.029443] LustreError: 323892:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [11441.140751] LustreError: 322840:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11441.162542] LustreError: 322840:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11442.044548] Lustre: server umount lustre-OST0000 complete [11446.240552] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11446.247216] Lustre: Skipped 1 previous similar message [11451.909952] Lustre: server umount lustre-MDT0000 complete [11454.520427] LustreError: 321001:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784198288 with bad export cookie 9759221034250547019 [11454.524288] LustreError: 321001:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11454.524755] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11454.839379] Lustre: server umount lustre-MDT0001 complete [11463.459199] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing unload_modules_local [11465.780533] Key type lgssc unregistered [11466.027099] LNet: 326535:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11466.034744] LNetError: 326535:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11466.066209] LNet: Removed LNI 192.168.201.108@tcp [11466.816753] Key type .llcrypt unregistered [11466.819589] Key type ._llcrypt unregistered [11486.338826] Key type ._llcrypt registered [11486.340498] Key type .llcrypt registered [11486.402688] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [11487.145114] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11487.194553] alg: No test for adler32 (adler32-zlib) [11488.315610] Lustre: Lustre: Build Version: 2.17.54_162_g9958c38 [11488.481559] LNet: Added LNI 192.168.201.108@tcp [8/256/0/180] [11490.111686] Key type lgssc registered [11491.001894] Lustre: Echo OBD driver; http://www.lustre.org/ [11497.538725] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11497.552260] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11498.800865] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11502.193777] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [11507.796834] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11508.194633] LustreError: 327933:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11508.211514] LustreError: 327933:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11513.005409] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11513.302322] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11516.965839] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [11523.743308] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11523.992500] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11529.013106] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [11535.255889] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11538.280606] LustreError: 329612:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11540.278653] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11540.416447] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11543.476136] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [11543.482857] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [11544.499514] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [11550.366609] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11556.336351] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [11556.461324] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11651.044540] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11651.051224] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11651.064309] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11651.071305] Lustre: Skipped 1 previous similar message [11656.162687] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11661.292875] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11661.300366] Lustre: Skipped 2 previous similar messages [11665.376553] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11665.435328] Lustre: server umount lustre-OST0001 complete [11666.407498] LustreError: 330872:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11666.425343] LustreError: 330872:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11671.530235] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11671.535237] LustreError: 329612:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11671.536986] LustreError: Skipped 1 previous similar message [11671.537079] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11671.537083] Lustre: Skipped 1 previous similar message [11671.542848] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11671.553931] LustreError: 329612:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11671.592493] Lustre: Skipped 1 previous similar message [11675.303119] Lustre: server umount lustre-OST0000 complete [11681.760872] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11681.777201] Lustre: Skipped 1 previous similar message [11681.787090] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11681.794867] Lustre: Skipped 1 previous similar message [11685.961690] Lustre: server umount lustre-MDT0000 complete [11686.882135] LustreError: 327929:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11686.893184] LustreError: 327929:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11688.706563] LustreError: 327914:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784198522 with bad export cookie 10953694674091830623 [11688.711477] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11688.719620] LustreError: 327914:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11688.949149] Lustre: server umount lustre-MDT0001 complete [11696.458860] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing unload_modules_local [11698.716916] Key type lgssc unregistered [11698.929721] LNet: 332875:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11698.937070] LNetError: 332875:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11698.949896] LNet: Removed LNI 192.168.201.108@tcp [11699.635468] Key type .llcrypt unregistered [11699.637936] Key type ._llcrypt unregistered [11732.484582] Key type ._llcrypt registered [11732.485781] Key type .llcrypt registered [11732.556219] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [11733.320856] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11733.369148] alg: No test for adler32 (adler32-zlib) [11734.315361] Lustre: Lustre: Build Version: 2.17.54_162_g9958c38 [11734.479595] LNet: Added LNI 192.168.201.108@tcp [8/256/0/180] [11736.103514] Key type lgssc registered [11736.967210] Lustre: Echo OBD driver; http://www.lustre.org/ [11743.345876] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11743.368939] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11744.714955] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11747.700789] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [11753.543790] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11753.763332] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11758.005486] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [11761.322140] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11768.627358] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11768.868489] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11769.925078] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11770.138405] LustreError: 336634:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11770.183119] LustreError: 336634:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11774.950799] LustreError: 336635:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11776.028979] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [11778.035987] LustreError: 336636:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11778.046973] LustreError: 336636:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11783.140619] LustreError: 336725:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11783.186891] LustreError: 336725:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11784.084566] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11784.320261] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11789.271481] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [11789.810191] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11796.388305] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11802.091487] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11810.273421] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11810.273797] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11810.293808] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11810.294249] Lustre: Skipped 2 previous similar messages [11810.301873] Lustre: Skipped 1 previous similar message [11815.406393] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11815.414454] Lustre: Skipped 3 previous similar messages [11816.252250] Lustre: server umount lustre-MDT0000 complete [11818.955321] LustreError: 338151:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784198653 with bad export cookie 7368274253958027832 [11818.957460] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11818.969158] LustreError: 338151:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11819.333328] Lustre: server umount lustre-MDT0001 complete [11832.440154] Lustre: server umount lustre-OST0000 complete [11844.815566] Lustre: server umount lustre-OST0001 complete [11854.723481] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [11860.490049] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11860.687534] LustreError: 340405:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11860.730822] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11863.112446] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [11867.926616] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11869.998356] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [11871.428973] Lustre: 341546:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11875.596450] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11875.855804] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11875.858768] Lustre: Skipped 1 previous similar message [11877.930183] LustreError: 341902:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11877.940324] LustreError: 341902:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11880.121905] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [11885.040336] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11885.491774] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11889.601527] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [11890.676488] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11895.595771] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11898.161694] Lustre: 343412:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11900.901664] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11900.904416] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11900.915744] Lustre: Skipped 4 previous similar messages [11900.929324] Lustre: Skipped 3 previous similar messages [11906.017900] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11906.027462] Lustre: Skipped 3 previous similar messages [11906.898600] Lustre: server umount lustre-MDT0000 complete [11909.237729] LustreError: 340386:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784198743 with bad export cookie 7368274253958029533 [11909.242070] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11909.248474] LustreError: 340386:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11909.465146] Lustre: server umount lustre-MDT0001 complete [11922.501763] Lustre: server umount lustre-OST0000 complete [11935.517210] Lustre: server umount lustre-OST0001 complete [11937.762074] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11938.880258] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11940.090630] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 06:46:14 (1784198774) [11957.106935] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [11963.008570] loop0: detected capacity change from 0 to 1099511627776 [11963.512752] SGI XFS with ACLs, security attributes, quota, no debug enabled [11963.521881] *** VALIDATE xfs *** [11963.529887] XFS (loop0): Mounting V5 Filesystem [11963.561270] XFS (loop0): Ending clean mount [11964.393961] loop1: detected capacity change from 0 to 18691697672192 [12025.033234] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_hostid [12031.895168] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [12056.139813] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12056.300222] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12056.315721] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12056.364877] Lustre: lustre-MDT0000: new disk, initializing [12056.446224] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12056.449544] Lustre: Skipped 1 previous similar message [12056.464853] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12059.185895] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [12066.335263] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12066.432724] Lustre: 348534:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12066.470859] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12066.474194] Lustre: Skipped 1 previous similar message [12066.534780] Lustre: lustre-MDT0001: new disk, initializing [12066.618272] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12066.622653] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12069.715652] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [12076.491543] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12080.630718] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12085.059874] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12085.259724] Lustre: lustre-OST0000: new disk, initializing [12085.261906] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12085.267175] Lustre: 350503:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12086.567268] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12086.574491] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12086.672803] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12089.257545] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [12095.824039] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12098.617325] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12098.757467] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12101.495811] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [12101.664776] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12107.232167] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12107.235861] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12107.238095] LustreError: Skipped 1 previous similar message [12107.251401] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12108.962778] Lustre: server umount lustre-OST0000 complete [12112.361256] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12112.371881] Lustre: Skipped 2 previous similar messages [12117.474234] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12117.481183] Lustre: Skipped 4 previous similar messages [12117.742238] Lustre: server umount lustre-MDT0000 complete [12120.432354] LustreError: 348527:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784198954 with bad export cookie 7368274253958031080 [12120.434323] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12120.446248] LustreError: 348527:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12120.778882] Lustre: server umount lustre-MDT0001 complete [12126.964604] XFS (loop0): Unmounting Filesystem [12130.413858] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 06:49:24 (1784198964) [12134.875124] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12135.185846] LustreError: 353056:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12135.196993] LustreError: 353056:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [12135.238906] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12135.243512] Lustre: Skipped 2 previous similar messages [12138.489497] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [12144.148770] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12147.479593] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [12152.955531] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12156.523980] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12160.963364] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12165.132713] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [12169.425537] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12190.177492] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12190.183245] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12190.195949] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12196.181441] Lustre: server umount lustre-OST0000 complete [12200.930810] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12200.935317] LustreError: Skipped 1 previous similar message [12200.938066] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12200.954458] Lustre: Skipped 1 previous similar message [12204.554880] Lustre: server umount lustre-MDT0000 complete [12206.560307] LustreError: 353051:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12206.570892] LustreError: 353051:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12206.728308] LustreError: 353039:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784199040 with bad export cookie 7368274253958031962 [12206.729747] LustreError: MGC192.168.201.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12206.732965] LustreError: 353039:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [12206.922714] Lustre: server umount lustre-MDT0001 complete [12213.669792] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing unload_modules_local [12215.518098] Key type lgssc unregistered [12215.697575] LNet: 356704:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12215.703145] LNetError: 356704:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12216.741372] LNet: Removed LNI 192.168.201.108@tcp [12217.364155] Key type .llcrypt unregistered [12217.368573] Key type ._llcrypt unregistered [12235.284675] Key type ._llcrypt registered [12235.286743] Key type .llcrypt registered [12235.402487] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing load_modules_local [12236.110996] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12236.125780] alg: No test for adler32 (adler32-zlib) [12237.121588] Lustre: Lustre: Build Version: 2.17.54_162_g9958c38 [12237.328429] LNet: Added LNI 192.168.201.108@tcp [8/256/0/180] [12239.015170] Key type lgssc registered [12239.652744] Lustre: Echo OBD driver; http://www.lustre.org/ [12244.827832] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12244.841212] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12246.157721] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12249.311450] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [12254.473304] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12254.710950] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12257.490504] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [12261.950144] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12264.759920] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12268.566974] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12268.779648] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12272.575215] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [12276.739825] Lustre: DEBUG MARKER: oleg108-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12284.386465] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12284.395983] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12284.399015] Lustre: Skipped 1 previous similar message [12288.666345] Lustre: server umount lustre-MDT0001 complete [12289.504555] LustreError: 358249:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12289.522351] LustreError: 358249:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [12294.624924] LustreError: 359167:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12294.646760] LustreError: 359167:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12295.242188] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12295.313648] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12295.315723] Lustre: Found index 1 for lustre-MDT0001, updating log [12295.338236] Lustre: 358237:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12295.493454] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12297.387477] Lustre: DEBUG MARKER: oleg108-server.virtnet: executing set_default_debug -1 all [12300.777039] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12301.811161] LustreError: 361903:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12301.818204] LustreError: 361903:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12301.829038] LustreError: 358240:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12301.877416] LustreError: 361914:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.201.108@tcp: cfg command failed: rc = -17 [12301.893949] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [12301.893949]