[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 382498994 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001010] APIC: Switch to symmetric I/O mode setup [ 0.002234] x2apic enabled [ 0.003005] Switched APIC routing to physical x2apic. [ 0.004006] kvm-guest: setup PV IPIs [ 0.006285] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007010] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008005] pid_max: default: 32768 minimum: 301 [ 0.009087] LSM: Security Framework initializing [ 0.010024] Yama: becoming mindful. [ 0.011018] SELinux: Initializing. [ 0.012058] *** VALIDATE selinux *** [ 0.018617] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.022541] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.023091] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.024059] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.025073] *** VALIDATE tmpfs *** [ 0.026303] *** VALIDATE proc *** [ 0.027177] *** VALIDATE cgroup *** [ 0.028005] *** VALIDATE cgroup2 *** [ 0.029223] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.030118] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.031003] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.032015] Spectre V2 : User space: Vulnerable [ 0.033004] Speculative Store Bypass: Vulnerable [ 0.035620] debug: unmapping init [mem 0xffffffff9d059000-0xffffffff9d060fff] [ 0.037833] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.038381] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.039008] ... version: 2 [ 0.039910] ... bit width: 48 [ 0.040006] ... generic registers: 4 [ 0.040766] ... value mask: 0000ffffffffffff [ 0.041006] ... max period: 00007fffffffffff [ 0.042008] ... fixed-purpose events: 3 [ 0.043006] ... event mask: 000000070000000f [ 0.044178] rcu: Hierarchical SRCU implementation. [ 0.046205] smp: Bringing up secondary CPUs ... [ 0.047381] x86: Booting SMP configuration: [ 0.048027] .... node #0, CPUs: #1 #2 #3 [ 0.050091] smp: Brought up 1 node, 4 CPUs [ 0.052007] smpboot: Max logical packages: 1 [ 0.053028] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.134306] node 0 deferred pages initialised in 80ms [ 0.137145] devtmpfs: initialized [ 0.138112] x86/mm: Memory block size: 128MB [ 0.140093] gcov: version magic: 0x41383552 [ 0.141479] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.142052] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.143150] pinctrl core: initialized pinctrl subsystem [ 0.144110] [ 0.144409] ************************************************************* [ 0.145006] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.146007] ** ** [ 0.147005] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.148005] ** ** [ 0.149005] ** This means that this kernel is built to expose internal ** [ 0.150005] ** IOMMU data structures, which may compromise security on ** [ 0.151006] ** your system. ** [ 0.152006] ** ** [ 0.153006] ** If you see this message and you are not debugging the ** [ 0.154006] ** kernel, report this immediately to your vendor! ** [ 0.155005] ** ** [ 0.156007] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.157006] ************************************************************* [ 0.158453] NET: Registered protocol family 16 [ 0.159290] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.160034] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.161026] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.162341] cpuidle: using governor menu [ 0.163360] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.165486] PCI: Using configuration type 1 for base access [ 0.167126] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.175056] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.176010] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.177100] cryptd: max_cpu_qlen set to 1000 [ 0.178177] ACPI: Added _OSI(Module Device) [ 0.180082] ACPI: Added _OSI(Processor Device) [ 0.181008] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.182006] ACPI: Added _OSI(Processor Aggregator Device) [ 0.185057] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.189491] ACPI: Interpreter enabled [ 0.191043] ACPI: PM: (supports S0 S3 S4 S5) [ 0.191977] ACPI: Using IOAPIC for interrupt routing [ 0.193052] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.195294] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.203758] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.205015] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.207011] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.209072] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.213249] acpiphp: Slot [2] registered [ 0.214070] acpiphp: Slot [5] registered [ 0.214886] acpiphp: Slot [6] registered [ 0.216109] acpiphp: Slot [7] registered [ 0.217091] acpiphp: Slot [8] registered [ 0.218088] acpiphp: Slot [9] registered [ 0.220069] acpiphp: Slot [10] registered [ 0.221000] acpiphp: Slot [3] registered [ 0.221000] acpiphp: Slot [4] registered [ 0.221000] acpiphp: Slot [11] registered [ 0.225103] acpiphp: Slot [12] registered [ 0.226008] acpiphp: Slot [13] registered [ 0.226863] acpiphp: Slot [14] registered [ 0.227056] acpiphp: Slot [15] registered [ 0.227984] acpiphp: Slot [16] registered [ 0.229042] acpiphp: Slot [17] registered [ 0.229891] acpiphp: Slot [18] registered [ 0.231047] acpiphp: Slot [19] registered [ 0.231865] acpiphp: Slot [20] registered [ 0.233038] acpiphp: Slot [21] registered [ 0.233886] acpiphp: Slot [22] registered [ 0.234037] acpiphp: Slot [23] registered [ 0.234895] acpiphp: Slot [24] registered [ 0.236053] acpiphp: Slot [25] registered [ 0.236959] acpiphp: Slot [26] registered [ 0.238054] acpiphp: Slot [27] registered [ 0.239113] acpiphp: Slot [28] registered [ 0.240047] acpiphp: Slot [29] registered [ 0.241052] acpiphp: Slot [30] registered [ 0.242056] acpiphp: Slot [31] registered [ 0.242947] PCI host bridge to bus 0000:00 [ 0.244009] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.246010] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.247008] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.249031] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.251008] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.253011] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.254134] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.256850] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.258957] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.267011] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.271378] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.273008] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.275008] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.277008] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.278471] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.280683] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.283026] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.285674] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.290008] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.300012] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.304009] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.308692] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.312939] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.318009] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.329011] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.337025] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.343010] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.349007] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.360961] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.370007] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.374009] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.379010] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.390758] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.398157] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.403010] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.408010] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.421010] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.429321] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.433827] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.438007] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.448857] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.455050] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.459672] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.463011] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.473921] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.482386] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.484335] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.486252] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.488204] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.490169] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.494302] iommu: Default domain type: Passthrough [ 0.496515] SCSI subsystem initialized [ 0.498132] ACPI: bus type USB registered [ 0.500088] usbcore: registered new interface driver usbfs [ 0.501031] usbcore: registered new interface driver hub [ 0.502042] usbcore: registered new device driver usb [ 0.503139] pps_core: LinuxPPS API ver. 1 registered [ 0.505006] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.507048] PTP clock support registered [ 0.509043] EDAC MC: Ver: 3.0.0 [ 0.510328] PCI: Using ACPI for IRQ routing [ 0.511675] NetLabel: Initializing [ 0.513011] NetLabel: domain hash size = 128 [ 0.514007] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.516064] NetLabel: unlabeled traffic allowed by default [ 0.518117] vgaarb: loaded [ 0.519220] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.520007] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.525290] clocksource: Switched to clocksource kvm-clock [ 0.613701] VFS: Disk quotas dquot_6.6.0 [ 0.614873] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.617110] *** VALIDATE ramfs *** [ 0.618189] *** VALIDATE hugetlbfs *** [ 0.619484] pnp: PnP ACPI init [ 0.622188] pnp: PnP ACPI: found 6 devices [ 0.635855] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.638472] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.640178] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.641970] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.643891] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.645960] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.648097] NET: Registered protocol family 2 [ 0.650080] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.654242] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.657049] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.661738] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.664711] TCP: Hash tables configured (established 65536 bind 65536) [ 0.666941] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.669223] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.671291] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.673641] NET: Registered protocol family 1 [ 0.675578] RPC: Registered named UNIX socket transport module. [ 0.677198] RPC: Registered udp transport module. [ 0.678635] RPC: Registered tcp transport module. [ 0.680230] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.682255] NET: Registered protocol family 44 [ 0.683862] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.685947] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.687473] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.689429] PCI: CLS 0 bytes, default 64 [ 0.691582] Unpacking initramfs... [ 1.991750] debug: unmapping init [mem 0xffff8a073cc54000-0xffff8a073ffbffff] [ 1.996311] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 1.998802] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.001195] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.472047] Initialise system trusted keyrings [ 2.473983] Key type blacklist registered [ 2.475495] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.484102] zbud: loaded [ 2.486460] *** VALIDATE nfs *** [ 2.487719] *** VALIDATE nfs4 *** [ 2.489201] pstore: using deflate compression [ 2.493701] Platform Keyring initialized [ 2.597951] NET: Registered protocol family 38 [ 2.599811] Key type asymmetric registered [ 2.601160] Asymmetric key parser 'x509' registered [ 2.602845] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.606497] io scheduler mq-deadline registered [ 2.608748] io scheduler kyber registered [ 2.610817] io scheduler bfq registered [ 2.612710] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.615915] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.618717] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.621862] ACPI: Power Button [PWRF] [ 2.627699] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.634955] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.649281] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 2.660001] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 2.683703] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.717768] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.757661] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 2.762349] Non-volatile memory driver v1.3 [ 2.764816] Linux agpgart interface v0.103 [ 2.793525] virtio_blk virtio1: [vda] 146656 512-byte logical blocks (75.1 MB/71.6 MiB) [ 2.796377] vda: detected capacity change from 0 to 75087872 [ 2.811601] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 2.815169] vdb: detected capacity change from 0 to 1073741824 [ 2.828421] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 2.830198] vdc: detected capacity change from 0 to 2621440000 [ 2.841213] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 2.844460] vdd: detected capacity change from 0 to 2621440000 [ 2.858917] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 2.861632] vde: detected capacity change from 0 to 4294967296 [ 2.877317] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 2.879967] vdf: detected capacity change from 0 to 4294967296 [ 2.888300] libphy: Fixed MDIO Bus: probed [ 2.894645] usbcore: registered new interface driver usbserial_generic [ 2.897568] usbserial: USB Serial support registered for generic [ 2.900554] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 2.905533] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 2.910389] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 2.912527] mousedev: PS/2 mouse device common for all mice [ 2.915436] rtc_cmos 00:05: RTC can wake from S4 [ 2.918853] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 2.919934] rtc_cmos 00:05: registered as rtc0 [ 2.925929] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 2.929975] intel_pstate: CPU model not supported [ 2.937527] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 2.940329] hid: raw HID events driver (C) Jiri Kosina [ 2.945416] usbcore: registered new interface driver usbhid [ 2.948385] usbhid: USB HID core driver [ 2.950167] drop_monitor: Initializing network drop monitor service [ 2.953366] Initializing XFRM netlink socket [ 2.955414] NET: Registered protocol family 10 [ 2.958795] Segment Routing with IPv6 [ 2.960260] NET: Registered protocol family 17 [ 2.960752] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 2.962805] mpls_gso: MPLS GSO support [ 2.971168] RAS: Correctable Errors collector initialized. [ 2.973783] AVX version of gcm_enc/dec engaged. [ 2.974880] AES CTR mode by8 optimization enabled [ 3.057098] sched_clock: Marking stable (3057083522, 0)->(3700756752, -643673230) [ 3.061402] registered taskstats version 1 [ 3.065423] Loading compiled-in X.509 certificates [ 3.069190] zswap: loaded using pool lzo/zbud [ 3.103770] Key type big_key registered [ 3.122943] Key type encrypted registered [ 3.125175] ima: No TPM chip found, activating TPM-bypass! [ 3.127186] ima: Allocated hash algorithm: sha1 [ 3.129012] ima: No architecture policies found [ 3.131533] evm: Initialising EVM extended attributes: [ 3.137016] evm: security.selinux [ 3.140197] evm: security.ima [ 3.143207] evm: security.capability [ 3.145101] evm: HMAC attrs: 0x1 [ 3.149539] rtc_cmos 00:05: setting system clock to 2026-09-13 01:39:16 UTC (1789263556) [ 3.157364] debug: unmapping init [mem 0xffffffff9e003000-0xffffffff9e1fffff] [ 3.160811] debug: unmapping init [mem 0xffffffff9cd82000-0xffffffff9d058fff] [ 3.170068] Write protecting the kernel read-only data: 28672k [ 3.173555] debug: unmapping init [mem 0xffffffff9b403000-0xffffffff9b5fffff] [ 3.176550] debug: unmapping init [mem 0xffffffff9bd14000-0xffffffff9bdfffff] [ 3.216207] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.229481] systemd[1]: Detected virtualization kvm. [ 3.233407] systemd[1]: Detected architecture x86-64. [ 3.235664] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.268178] systemd[1]: No hostname configured. [ 3.271088] systemd[1]: Set hostname to . [ 3.274540] random: systemd: uninitialized urandom read (16 bytes read) [ 3.280494] systemd[1]: Initializing machine ID from random generator. [ 3.429376] random: systemd: uninitialized urandom read (16 bytes read) [ 3.432799] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ 3.438410] random: systemd: uninitialized urandom read (16 bytes read) [ 3.441413] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 3.452146] systemd[1]: Starting Journal Service... Starting Journal Service... [ OK ] Reached target Local File Systems. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Slices. [ OK ] Reached target Initrd Root Device. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. Starting Setup Virtual Console... [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. Starting Apply Kernel Variables... Starting Create Volatile Files and Directories... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Timers. [ OK ] Reached target Swap. [ OK ] Started Memstrack Anylazing Service. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.364270] device-mapper: uevent: version 1.0.3 [ 4.366075] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ 5.215901] random: fast init done [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.262129] virtio_net virtio0 ens2: renamed from eth0 [ 5.382499] scsi host0: ata_piix [ 5.437403] scsi host1: ata_piix [ 5.443037] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.447391] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.664587] random: crng init done [ 9.666869] random: 7 urandom warning(s) missed due to ratelimiting [ 10.423734] dracut-initqueue[586]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 12.431972] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Timers. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Swap. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 14.740073] printk: systemd: 24 output lines suppressed due to ratelimiting [ 15.258254] SELinux: Disabled at runtime. [ 15.351584] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 15.366538] systemd[1]: Detected virtualization kvm. [ 15.368823] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 16.203962] systemd[1]: initrd-switch-root.service: Succeeded. [ 16.207225] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 16.218553] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 16.225496] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 16.229581] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 16.237539] systemd[1]: Starting Journal Service... Starting Journal Service... [ 16.255078] systemd[1]: Starting Remount Root and Kernel File Systems... Starting Remount Root and Kernel File Systems... [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Mounting Kernel Debug File System... [ OK ] Listening on udev Kernel Socket. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Listening on Process Core Dump Socket. [ OK ] Reached target rpc_pipefs.target. Starting Create list of required st…ce nodes for the current kernel... Activating swap /dev/disk/by-label/SWAP... Mounting Huge Pages File System... Mounting POSIX Message Queue File System... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Stopped target Initrd File Systems. [ OK ] Created slice system-getty.slice. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Listening on initctl Compatibility Named Pipe. [ 16.611534] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Starting Apply Kernel Variables... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Reached target Paths. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Journal Service. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Apply Kernel Variables. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 17.435245] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 18.091941] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 18.347342] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 18.866398] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 19.049513] EDAC sbridge: Ver: 1.1.2 [ 21.277926] Key type dns_resolver registered [ 21.690808] NFS: Registering the id_resolver key type [ 21.692908] Key type id_resolver registered [ 21.696287] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Mark the need to relabel after reboot... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started dnf makecache --timer. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Reached target Basic System. [ OK ] Started D-Bus System Message Bus. Starting Restore /run/initramfs on shutdown... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Network Manager... Starting Login Service... [ OK ] Started irqbalance daemon. [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started Login Service. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. Starting Authorization Manager... Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg315-server login: [ 63.393374] libcfs: loading out-of-tree module taints kernel. [ 63.419037] Key type ._llcrypt registered [ 63.420625] Key type .llcrypt registered [ 63.489399] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_hostid [ 72.536313] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 73.279787] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 73.289177] alg: No test for adler32 (adler32-zlib) [ 74.355636] Lustre: Lustre: Build Version: 2.17.57_103_g3c4c45f [ 74.732418] LNet: Added LNI 192.168.203.115@tcp [8/256/0/180] [ 76.368211] Key type lgssc registered [ 77.071690] Lustre: Echo OBD driver; http://www.lustre.org/ [ 84.346064] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 102.494613] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 107.852685] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 107.864850] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 108.974927] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 108.993931] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 109.039415] Lustre: lustre-MDT0000: new disk, initializing [ 109.072434] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 109.081774] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 110.631429] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 116.608756] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 116.650773] Lustre: 6486:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 116.666583] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 116.669864] Lustre: Skipped 1 previous similar message [ 116.710879] Lustre: lustre-MDT0001: new disk, initializing [ 116.735655] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 116.746662] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 116.752415] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 118.369423] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 120.880445] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 124.689643] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 124.803624] Lustre: lustre-OST0000: new disk, initializing [ 124.806092] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 124.808870] Lustre: 8423:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 124.835506] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 126.219671] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 126.226603] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 126.269265] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 127.223519] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 133.425105] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 133.475085] Lustre: lustre-OST0001: new disk, initializing [ 133.478367] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 133.481826] Lustre: 9495:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 133.509272] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 135.914970] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 142.322595] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 142.331067] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 142.364325] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 142.564323] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 149.603284] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 152.127616] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing check_logdir /tmp/testlogs/ [ 154.192018] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing yml_node [ 156.158721] Lustre: DEBUG MARKER: Client: 2.17.57.103 [ 157.206451] Lustre: DEBUG MARKER: MDS: 2.17.57.103 [ 158.292340] Lustre: DEBUG MARKER: OSS: 2.17.57.103 [ 158.997232] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Sat Sep 12 21:41:51 EDT 2026 [ 166.126961] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 166.829674] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 183.265542] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 183.265809] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 183.267057] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 183.271421] Lustre: Skipped 1 previous similar message [ 184.937214] Lustre: server umount lustre-MDT0000 complete [ 186.338887] LustreError: 6478:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789263739 with bad export cookie 2157251684326473572 [ 186.340942] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 186.344410] LustreError: 6478:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 188.384830] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 188.385371] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 188.390854] Lustre: Skipped 3 previous similar messages [ 188.397024] Lustre: Skipped 4 previous similar messages [ 191.392541] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 191.395280] Lustre: Skipped 1 previous similar message [ 192.485343] Lustre: server umount lustre-MDT0001 complete [ 193.907951] Lustre: server umount lustre-OST0000 complete [ 195.296656] Lustre: server umount lustre-OST0001 complete [ 197.658202] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_hostid [ 201.100725] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 214.324186] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 214.423809] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 214.438275] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 214.478273] Lustre: lustre-MDT0000: new disk, initializing [ 214.503756] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 214.510545] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 216.039488] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 220.634089] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 220.685762] Lustre: 15049:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 220.805947] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 222.336628] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 226.359392] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 228.523779] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 230.944486] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 231.062526] Lustre: lustre-OST0000: new disk, initializing [ 231.064540] Lustre: Skipped 1 previous similar message [ 231.067087] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 231.070109] Lustre: Skipped 2 previous similar messages [ 231.073324] Lustre: 17016:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 231.108426] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 231.112458] Lustre: Skipped 1 previous similar message [ 232.958260] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 232.961826] Lustre: Skipped 1 previous similar message [ 232.964101] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 233.005086] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 233.252242] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 237.301392] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 238.880255] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 238.976909] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 240.451117] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 240.530797] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 243.172042] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 243.173528] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 243.174474] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 243.174652] LustreError: Skipped 1 previous similar message [ 243.185657] Lustre: Skipped 1 previous similar message [ 247.327401] Lustre: server umount lustre-OST0000 complete [ 251.873213] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 251.877082] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 251.885189] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 251.889114] Lustre: Skipped 1 previous similar message [ 263.136232] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 263.208184] Lustre: server umount lustre-MDT0000 complete [ 263.649105] LustreError: 16040:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 263.656518] LustreError: 16040:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 264.631891] LustreError: 15040:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789263817 with bad export cookie 2157251684326475091 [ 264.637534] LustreError: 15040:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 264.637950] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 264.767600] Lustre: server umount lustre-MDT0001 complete [ 266.665744] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 21:43:39 (1789263819) [ 269.198243] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 269.397039] LustreError: 19278:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 269.430319] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 271.035762] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 274.344256] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 276.100210] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 278.884195] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 280.983390] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 283.448940] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 285.945756] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 288.806673] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 301.627441] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 305.123489] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 305.125436] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 305.129536] Lustre: Skipped 3 previous similar messages [ 305.130547] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 305.132318] Lustre: Skipped 6 previous similar messages [ 311.074506] Lustre: server umount lustre-MDT0000 complete [ 312.525879] LustreError: 20015:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789263865 with bad export cookie 2157251684326475973 [ 312.527850] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 312.532822] LustreError: 20015:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 312.644789] Lustre: server umount lustre-MDT0001 complete [ 324.454536] Lustre: server umount lustre-OST0000 complete [ 332.123935] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 335.845818] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 335.984914] LustreError: 23677:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 335.999298] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 336.001601] Lustre: Skipped 2 previous similar messages [ 337.382172] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 340.309220] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 341.901524] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 342.977789] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 345.362041] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 347.658648] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 350.710361] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 350.777427] Lustre: lustre-OST0001: new disk, initializing [ 350.778928] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 350.781617] Lustre: 25992:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 352.071791] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 352.075965] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 352.109221] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 352.805577] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 357.347908] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 360.294273] Lustre: Setting parameter general.obdfilter.*.client_cache_count=256 in log params [ 360.297062] Lustre: Skipped 1 previous similar message [ 362.465990] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 362.469093] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 362.471167] Lustre: Skipped 3 previous similar messages [ 362.474363] Lustre: Skipped 5 previous similar messages [ 368.712452] Lustre: server umount lustre-MDT0000 complete [ 371.568162] LustreError: 23658:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789263924 with bad export cookie 2157251684326476820 [ 371.572190] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 371.583370] LustreError: 23658:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 371.832972] Lustre: server umount lustre-MDT0001 complete [ 385.142368] Lustre: server umount lustre-OST0000 complete [ 398.772959] Lustre: server umount lustre-OST0001 complete [ 427.644392] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 440.214372] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 440.621352] LustreError: 28977:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 440.694927] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 440.701275] Lustre: Skipped 3 previous similar messages [ 443.876367] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 445.921957] LustreError: 28978:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 450.167825] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 453.898936] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 456.180726] Lustre: 30123:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 460.641339] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 465.010377] LustreError: 30476:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 465.027837] LustreError: 30476:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 465.074145] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 470.742786] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 474.404798] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 479.238555] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 481.198591] Lustre: 32001:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 485.350295] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 485.353170] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 485.359507] Lustre: Skipped 3 previous similar messages [ 485.361769] Lustre: Skipped 6 previous similar messages [ 490.577408] Lustre: server umount lustre-MDT0000 complete [ 492.797641] LustreError: 28959:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789264046 with bad export cookie 2157251684326478052 [ 492.800837] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 492.806261] LustreError: 28959:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 493.012770] Lustre: server umount lustre-MDT0001 complete [ 506.040925] Lustre: server umount lustre-OST0000 complete [ 518.669481] Lustre: server umount lustre-OST0001 complete [ 523.556440] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 21:47:55 (1789264075) [ 539.221526] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 544.267639] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 544.477687] LustreError: 34800:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 544.488123] LustreError: 34800:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 546.776282] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 551.922112] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 554.883338] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 556.832673] Lustre: 35946:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 561.334372] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 566.678760] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 572.991183] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 573.192424] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 573.198571] Lustre: Skipped 6 previous similar messages [ 577.868187] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 584.101732] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 586.540398] Lustre: 37824:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 593.377649] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 593.387035] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 593.407255] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 593.410781] Lustre: Skipped 5 previous similar messages [ 595.838407] Lustre: server umount lustre-MDT0000 complete [ 598.517767] LustreError: 34780:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789264151 with bad export cookie 2157251684326479151 [ 598.522648] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 598.527337] LustreError: 34780:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 598.773760] Lustre: server umount lustre-MDT0001 complete [ 610.840551] Lustre: server umount lustre-OST0000 complete [ 622.982139] Lustre: server umount lustre-OST0001 complete [ 628.843194] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 21:49:40 (1789264180) [ 640.487309] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 648.163262] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 648.438620] LustreError: 39966:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 648.453015] LustreError: 39966:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 651.962252] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 658.616910] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 662.689900] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 665.295574] Lustre: 41116:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 671.200647] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 676.825200] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 683.981086] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 689.240267] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 695.650730] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 698.541635] Lustre: 42990:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 699.236686] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 704.996878] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 705.011422] Lustre: Skipped 4 previous similar messages [ 705.014015] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 707.815512] Lustre: server umount lustre-MDT0000 complete [ 710.458364] LustreError: 39947:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789264263 with bad export cookie 2157251684326480257 [ 710.465967] LustreError: 39947:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 710.468482] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 715.241741] LustreError: 39962:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 715.256733] LustreError: 39962:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 12 previous similar messages [ 716.739681] Lustre: server umount lustre-MDT0001 complete [ 722.594429] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 734.690369] LustreError: 3623:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff8a07bd895f80 x1876178898305280/t0(0) o250->MGC192.168.203.115@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 737.718949] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 742.943104] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 745.320984] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 746.125604] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 751.901380] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 753.024814] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 755.593539] Lustre: lustre-MDT0001-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 755.600829] Lustre: Skipped 1 previous similar message [ 757.693403] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 759.030800] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 763.889351] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 763.900065] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 763.905645] Lustre: Skipped 12 previous similar messages [ 769.598522] Lustre: server umount lustre-MDT0000 complete [ 772.557643] LustreError: 39946:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789264325 with bad export cookie 2157251684326481384 [ 772.566475] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 772.569043] LustreError: 39946:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 772.846942] Lustre: server umount lustre-MDT0001 complete [ 786.469913] Lustre: server umount lustre-OST0000 complete [ 799.460161] Lustre: server umount lustre-OST0001 complete [ 806.197040] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 21:52:37 (1789264357) [ 819.304737] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 826.659439] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 829.903309] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 836.224686] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 836.575288] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 836.578515] Lustre: Skipped 7 previous similar messages [ 840.475806] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 843.072107] Lustre: 48746:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 848.629359] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 852.071825] LustreError: 49098:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 852.085387] LustreError: 49098:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [ 854.397343] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 861.817071] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 867.522062] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 874.061890] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 878.254222] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 890.141450] Lustre: server umount lustre-MDT0000 complete [ 892.908161] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 892.908815] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 892.919227] Lustre: Skipped 10 previous similar messages [ 893.134802] LustreError: 47577:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789264446 with bad export cookie 2157251684326482182 [ 893.139864] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 893.147543] LustreError: 47577:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 893.377994] Lustre: server umount lustre-MDT0001 complete [ 906.345627] Lustre: server umount lustre-OST0000 complete [ 919.792805] Lustre: server umount lustre-OST0001 complete [ 926.839271] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 21:54:38 (1789264478) [ 928.872491] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 930.170622] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 21:54:42 (1789264482) [ 937.841694] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_hostid [ 944.208016] hrtimer: interrupt took 6008623 ns [ 944.682628] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 959.989414] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 960.179481] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 960.270850] Lustre: lustre-MDT0000: new disk, initializing [ 960.368888] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 963.256164] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 969.856886] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 974.660185] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 974.837116] Lustre: 54818:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 976.444712] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 976.465741] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 978.926369] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 985.916656] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 987.583343] Lustre: DEBUG MARKER: create test files [ 1007.079595] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1012.580710] Lustre: server umount lustre-OST0000 complete [ 1015.066401] Lustre: server umount lustre-MDT0000 complete [ 1021.702664] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing unload_modules_local [ 1023.562101] Key type lgssc unregistered [ 1023.772953] LNet: 56447:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1023.780511] LNetError: 56447:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1023.791927] LNet: Removed LNI 192.168.203.115@tcp [ 1024.467153] Key type .llcrypt unregistered [ 1024.470296] Key type ._llcrypt unregistered [ 1043.661200] Key type ._llcrypt registered [ 1043.664529] Key type .llcrypt registered [ 1043.756749] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 1044.522520] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1044.541791] alg: No test for adler32 (adler32-zlib) [ 1045.536514] Lustre: Lustre: Build Version: 2.17.57_103_g3c4c45f [ 1045.717152] LNet: Added LNI 192.168.203.115@tcp [8/256/0/180] [ 1047.336528] Key type lgssc registered [ 1048.072878] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1053.415414] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1053.428675] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1054.692352] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1057.480849] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 1062.760423] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1067.566951] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1067.865044] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1071.948569] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 1076.749650] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1080.179163] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:100 to 0x240000400:129) [ 1080.241318] Lustre: 59155:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1080.247611] Lustre: 59155:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1080.251128] Lustre: 59155:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1080.255243] Lustre: 59155:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1080.259488] Lustre: 59155:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1080.268558] Lustre: 59155:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1087.007184] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1090.530684] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1095.650438] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1095.651089] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1095.671646] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1100.776616] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1101.197669] Lustre: server umount lustre-OST0000 complete [ 1104.043651] Lustre: server umount lustre-MDT0000 complete [ 1112.119194] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing unload_modules_local [ 1114.353057] Key type lgssc unregistered [ 1114.604228] LNet: 60759:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1114.610410] LNetError: 60759:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1114.627847] LNet: Removed LNI 192.168.203.115@tcp [ 1115.321177] Key type .llcrypt unregistered [ 1115.323470] Key type ._llcrypt unregistered [ 1136.561943] Key type ._llcrypt registered [ 1136.563877] Key type .llcrypt registered [ 1136.638942] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 1137.342225] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1137.357271] alg: No test for adler32 (adler32-zlib) [ 1138.368808] Lustre: Lustre: Build Version: 2.17.57_103_g3c4c45f [ 1138.559020] LNet: Added LNI 192.168.203.115@tcp [8/256/0/180] [ 1140.176181] Key type lgssc registered [ 1141.058759] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1146.542276] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1146.564467] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1147.902970] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1151.101335] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 1156.718233] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1161.641909] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1161.906894] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1166.018352] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 1171.655421] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1173.163232] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1173.224923] Lustre: 63564:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 514 < left 544, rollback = 9 [ 1173.232333] Lustre: 63564:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/1, destroy: 0/0/0 [ 1173.248258] Lustre: 63564:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1173.256107] Lustre: 63564:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1173.261941] Lustre: 63564:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1173.267783] Lustre: 63564:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1181.015821] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1188.836176] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1188.844528] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1188.854660] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1193.348749] Lustre: server umount lustre-OST0000 complete [ 1196.178518] Lustre: server umount lustre-MDT0000 complete [ 1203.796416] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing unload_modules_local [ 1205.778285] Key type lgssc unregistered [ 1206.030468] LNet: 65120:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1206.037253] LNetError: 65120:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1207.084314] LNet: Removed LNI 192.168.203.115@tcp [ 1207.676086] Key type .llcrypt unregistered [ 1207.678948] Key type ._llcrypt unregistered [ 1221.225798] Key type ._llcrypt registered [ 1221.228721] Key type .llcrypt registered [ 1221.306473] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_hostid [ 1232.940662] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 1233.652135] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1233.729691] alg: No test for adler32 (adler32-zlib) [ 1234.716275] Lustre: Lustre: Build Version: 2.17.57_103_g3c4c45f [ 1234.882782] LNet: Added LNI 192.168.203.115@tcp [8/256/0/180] [ 1236.512218] Key type lgssc registered [ 1237.274575] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1262.702848] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1262.712124] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1263.876749] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1263.910653] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1263.969803] Lustre: lustre-MDT0000: new disk, initializing [ 1264.024481] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1264.042912] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1266.786575] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 1274.228308] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1274.328291] Lustre: 68245:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1274.380825] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1274.383917] Lustre: Skipped 1 previous similar message [ 1274.443579] Lustre: lustre-MDT0001: new disk, initializing [ 1274.507095] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1274.538357] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1274.547901] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1277.412333] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 1284.079223] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1288.544468] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1293.236327] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1293.446481] Lustre: lustre-OST0000: new disk, initializing [ 1293.449600] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1293.456805] Lustre: 70213:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1293.528648] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1297.661715] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 1299.475495] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1299.480822] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1299.554521] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1304.692236] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1308.314434] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1308.522767] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1311.825415] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1311.982823] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1314.793500] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1314.795266] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1314.795756] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1314.806433] Lustre: Skipped 1 previous similar message [ 1319.566226] Lustre: server umount lustre-OST0000 complete [ 1325.026051] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1325.027930] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1325.037021] Lustre: Skipped 1 previous similar message [ 1325.037518] Lustre: Skipped 1 previous similar message [ 1328.434518] Lustre: server umount lustre-MDT0000 complete [ 1331.145687] LustreError: 68239:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789264884 with bad export cookie 15324128047128777953 [ 1331.155831] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1331.157611] LustreError: 68239:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1331.368936] Lustre: server umount lustre-MDT0001 complete [ 1336.873771] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 22:01:28 (1789264888) [ 1343.319937] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 1367.202736] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_hostid [ 1373.396314] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 1395.115527] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1395.287405] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1395.308705] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1395.355894] Lustre: lustre-MDT0000: new disk, initializing [ 1395.464958] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1395.487296] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1400.253578] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 1414.534802] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1414.842688] Lustre: 76239:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1414.961500] LustreError: 76251:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1414.967936] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1414.999131] LustreError: 76251:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 1415.027287] Lustre: Skipped 1 previous similar message [ 1415.170494] Lustre: lustre-MDT0001: new disk, initializing [ 1415.375794] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1415.536038] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1415.559562] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1422.177433] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 1434.243491] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1442.408797] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1451.897138] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1452.201255] Lustre: lustre-OST0000: new disk, initializing [ 1452.204535] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1452.214318] Lustre: 78210:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1452.309423] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1458.871334] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 1460.432109] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1460.450817] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1460.704846] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1474.904723] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1481.024338] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1481.163919] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1485.638317] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1485.826365] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1491.433862] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1491.438146] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1491.443571] LustreError: Skipped 1 previous similar message [ 1491.444312] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1491.473944] Lustre: Skipped 1 previous similar message [ 1493.871257] Lustre: server umount lustre-OST0000 complete [ 1497.578879] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1497.590479] LustreError: Skipped 1 previous similar message [ 1497.603410] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1497.623522] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1497.628981] Lustre: Skipped 1 previous similar message [ 1501.666089] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1501.667453] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1501.679886] Lustre: Skipped 1 previous similar message [ 1503.716775] Lustre: server umount lustre-MDT0000 complete [ 1506.785804] LustreError: 76247:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1506.807966] LustreError: 76247:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 1507.367867] LustreError: 76233:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789265060 with bad export cookie 15324128047128778835 [ 1507.372649] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1507.381708] LustreError: 76233:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1507.769616] Lustre: server umount lustre-MDT0001 complete [ 1515.269725] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 22:04:26 (1789265066) [ 1521.991625] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1522.555380] LustreError: 80621:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1522.654590] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1527.066123] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 1527.779203] LustreError: 80622:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1531.888818] LustreError: 80621:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1534.762641] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1539.386209] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 1546.825524] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1553.060289] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1559.239331] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1559.565460] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1559.573160] Lustre: Skipped 1 previous similar message [ 1566.136987] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 1573.434836] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1576.825508] Lustre: 83290:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting fd20e68e-da64-4c3b-9994-7e253e43eb66 at adminstrative request [ 1576.953067] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1576.969344] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 1587.181352] Lustre: MGS: Client fd20e68e-da64-4c3b-9994-7e253e43eb66 (at 0@lo) reconnecting [ 1587.194047] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xd4aa3d44a5b61bc5 to 0xd4aa3d44a5b61e65 [ 1587.231184] Lustre: MGC192.168.203.115@tcp: Connection restored to 0@lo (at 0@lo) [ 1613.676239] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1613.812114] Lustre: lustre-OST0001: new disk, initializing [ 1613.815237] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1613.820966] Lustre: 83686:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1619.748867] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 1621.098096] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1621.115925] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1621.233158] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1628.629971] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1631.713693] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1631.729793] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1631.734014] LustreError: Skipped 1 previous similar message [ 1631.740474] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 1631.772505] Lustre: Skipped 1 previous similar message [ 1637.197291] Lustre: server umount lustre-OST0001 complete [ 1641.962753] LustreError: 83326:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1641.978987] LustreError: 83326:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 1652.222315] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1652.233375] Lustre: Skipped 8 previous similar messages [ 1655.264248] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1655.582906] Lustre: server umount lustre-OST0000 complete [ 1662.438842] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1662.461120] Lustre: Skipped 3 previous similar messages [ 1666.633584] Lustre: server umount lustre-MDT0000 complete [ 1667.553930] LustreError: 80622:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1667.581149] LustreError: 80622:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 1671.238296] LustreError: 80602:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789265224 with bad export cookie 15324128047128780389 [ 1671.244092] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1671.249375] LustreError: 80602:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1671.668253] Lustre: server umount lustre-MDT0001 complete [ 1680.432798] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 22:07:11 (1789265231) [ 1682.198417] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 1684.192681] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 22:07:15 (1789265235) [ 1685.860746] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 1687.599134] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 22:07:19 (1789265239) [ 1689.401981] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 1691.008153] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 22:07:22 (1789265242) [ 1692.348725] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 1693.936700] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 22:07:25 (1789265245) [ 1695.423329] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 1701.282837] loop: module loaded [ 1701.303236] loop0: detected capacity change from 0 to 4294967296 [ 1701.337502] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 1703.187568] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 1706.911498] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 1710.963840] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 1737.331908] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1737.373112] LustreError: 87812:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 1737.377614] LustreError: 87812:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 1737.382338] LustreError: 87812:0:(tgt_mount.c:2582:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 1737.388286] LustreError: 87812:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 1758.135838] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_hostid [ 1769.319464] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 1809.213613] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1809.507220] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1809.556690] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1809.699452] Lustre: lustre-MDT0000: new disk, initializing [ 1809.810062] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1809.818429] Lustre: Skipped 1 previous similar message [ 1809.839495] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1814.914353] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 1826.778577] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1826.917662] Lustre: 90678:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1827.186514] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1831.623908] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 1841.402034] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1847.718984] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1855.133643] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1855.457211] Lustre: 92647:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1861.912768] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 1862.713597] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1862.885576] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1871.932534] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1876.380452] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1876.621819] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1881.348261] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1881.575949] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1883.623233] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1883.626024] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1883.634461] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1883.634470] Lustre: Skipped 2 previous similar messages [ 1883.639464] LustreError: Skipped 1 previous similar message [ 1883.648388] Lustre: Skipped 1 previous similar message [ 1889.809263] Lustre: server umount lustre-OST0000 complete [ 1899.570933] Lustre: server umount lustre-MDT0000 complete [ 1903.230630] LustreError: 90671:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789265456 with bad export cookie 15324128047128781005 [ 1903.243138] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1903.244783] LustreError: 90671:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1903.761336] Lustre: server umount lustre-MDT0001 complete [ 1913.153225] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 22:11:04 (1789265464) [ 1920.186401] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1920.617129] LustreError: 95060:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1925.344328] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 1934.350771] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1939.177360] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 1946.762952] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1952.885632] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1959.825955] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1960.249146] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1960.255827] Lustre: Skipped 4 previous similar messages [ 1967.140914] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 1975.082932] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1980.332695] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1980.561053] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1985.657148] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1985.909955] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1992.661693] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1992.949742] Lustre: lustre-OST0001: new disk, initializing [ 1992.952285] Lustre: Skipped 2 previous similar messages [ 1992.964219] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1992.970578] Lustre: Skipped 3 previous similar messages [ 1992.983390] Lustre: 98431:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1999.482391] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 2000.017285] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2000.027556] Lustre: Skipped 2 previous similar messages [ 2000.042112] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2000.171414] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2009.163569] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2014.245346] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2014.541323] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2018.637248] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2018.863364] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2037.676442] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2064.717403] Lustre: Failing over lustre-MDT0000 [ 2065.026575] Lustre: server umount lustre-MDT0000 complete [ 2066.914602] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2066.918535] LustreError: 95055:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2066.928536] Lustre: Skipped 4 previous similar messages [ 2066.954350] LustreError: 95055:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 2083.296168] Lustre: 66469:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789265620/real 1789265620] req@00000000a9ce3dc9 x1876180114615936/t0(0) o400->MGC192.168.203.115@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1789265636 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2083.328945] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2090.226522] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2090.228621] LDISKFS-fs (dm-0): recovery complete [ 2090.240294] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2094.026203] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2098.192470] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2098.193404] Lustre: 101258:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2099.188994] Lustre: 95060:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2099.194859] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2099.212027] Lustre: 95060:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 2099.331081] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 2099.364162] LustreError: 101281:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2099.713924] LustreError: 101281:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2100.067820] LustreError: 101281:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2100.084343] LustreError: 101281:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2100.441066] LustreError: 101281:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2100.452332] LustreError: 101281:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2100.465694] Lustre: 101281:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2100.474410] Lustre: 101281:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 2101.141424] LustreError: 101281:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2101.155615] LustreError: 101281:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2101.472063] LustreError: 101281:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2101.486761] LustreError: 101281:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2102.495091] Lustre: 101281:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2102.520765] Lustre: 101281:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 5 previous similar messages [ 2103.169848] LustreError: 101281:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2103.197735] LustreError: 101281:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 5 previous similar messages [ 2103.520119] LustreError: 101281:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2103.525830] LustreError: 101281:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 5 previous similar messages [ 2106.699993] Lustre: 101281:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2106.710930] Lustre: 101281:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 2107.349047] LustreError: 101281:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2107.363761] LustreError: 101281:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2107.696189] LustreError: 101281:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2107.701234] LustreError: 101281:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2115.003394] Lustre: 101281:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2115.014702] Lustre: 101281:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 25 previous similar messages [ 2115.353802] LustreError: 101281:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2115.380910] LustreError: 101281:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2115.712510] LustreError: 101281:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2115.725037] LustreError: 101281:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2131.297911] Lustre: 101281:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2131.314404] Lustre: 101281:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 50 previous similar messages [ 2131.642884] LustreError: 101281:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2131.652669] LustreError: 101281:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2131.968140] LustreError: 101281:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2131.979459] LustreError: 101281:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2158.193572] Lustre: 101281:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2158.203537] Lustre: 101281:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2158.214277] Lustre: 101281:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 70a0f7e2-e83e-4a9b-9f9e-2c036240ffe4@192.168.203.15@tcp [ 2158.227326] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2158.238578] Lustre: 101281:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@000000003cb3aa8c x1876180101830656/t0(8589934780) o35->70a0f7e2-e83e-4a9b-9f9e-2c036240ffe4@192.168.203.15@tcp:282/0 lens 392/0 e 0 to 0 dl 1789265722 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2158.270538] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2158.286952] Lustre: lustre-MDT0000: Denying connection for new client 70a0f7e2-e83e-4a9b-9f9e-2c036240ffe4 (at 192.168.203.15@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2158.313731] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2158.378543] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2158.395462] Lustre: Skipped 2 previous similar messages [ 2158.401606] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2158.457268] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:48 to 0x2c0000401:545) [ 2158.457970] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:80 to 0x280000401:577) [ 2171.366552] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2171.387267] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2171.393726] Lustre: Skipped 7 previous similar messages [ 2177.246365] Lustre: server umount lustre-OST0000 complete [ 2187.083350] Lustre: server umount lustre-OST0001 complete [ 2205.153981] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2205.587497] Lustre: server umount lustre-MDT0000 complete [ 2206.693166] LustreError: 96082:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2206.717592] LustreError: 96082:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 43 previous similar messages [ 2209.656187] LustreError: 95042:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789265763 with bad export cookie 15324128047128832224 [ 2209.657615] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2209.667531] LustreError: 95042:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2209.981693] Lustre: server umount lustre-MDT0001 complete [ 2218.377287] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 22:16:09 (1789265769) [ 2226.992692] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2226.998767] Lustre: *** cfs_fail_loc=197, val=0*** [ 2242.593044] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2247.712452] LustreError: 103260:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.115@tcp: failed processing log, type 4: rc = -110 [ 2252.926366] Lustre: *** cfs_fail_loc=197, val=0*** [ 2252.929800] Lustre: Skipped 65 previous similar messages [ 2273.505886] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2273.516897] Lustre: Skipped 2 previous similar messages [ 2281.011079] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 2288.902402] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2291.186938] Lustre: server umount lustre-OST0000 complete [ 2300.899884] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 22:17:31 (1789265851) [ 2323.298629] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2328.460794] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 2338.623896] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2343.022201] LustreError: 104912:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2343.047331] LustreError: 104912:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2343.057449] LustreError: 104912:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 19, retries 0, failed: rc = -5 [ 2343.380298] Lustre: server umount lustre-MDT0000 complete [ 2379.820091] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_hostid [ 2389.255349] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 2426.994486] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2427.522975] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2427.597653] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2427.716207] Lustre: lustre-MDT0000: new disk, initializing [ 2427.876122] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2434.733539] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 2449.073182] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2449.192399] Lustre: 109090:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2449.548307] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2453.571309] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 2464.330949] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2471.013675] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2478.748871] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2478.974458] Lustre: 111060:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2485.100152] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 2488.450378] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2493.860915] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2498.669433] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2498.818982] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2503.512444] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2503.786739] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2508.773214] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2508.773947] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2508.789483] LustreError: Skipped 2 previous similar messages [ 2508.798503] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2508.798864] Lustre: Skipped 8 previous similar messages [ 2508.822153] Lustre: Skipped 14 previous similar messages [ 2511.779869] Lustre: server umount lustre-OST0000 complete [ 2529.248232] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2529.505829] Lustre: server umount lustre-MDT0000 complete [ 2533.887981] LustreError: 111059:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789266087 with bad export cookie 15324128047128842920 [ 2533.896292] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2533.904749] LustreError: 111059:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2534.392862] Lustre: server umount lustre-MDT0001 complete [ 2544.045423] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 22:21:35 (1789266095) [ 2552.692134] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing unload_modules_local [ 2555.516636] Key type lgssc unregistered [ 2555.877726] LNet: 113556:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2555.885110] LNetError: 113556:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2555.902369] LNet: Removed LNI 192.168.203.115@tcp [ 2557.002525] Key type .llcrypt unregistered [ 2557.008967] Key type ._llcrypt unregistered [ 2574.338995] Key type ._llcrypt registered [ 2574.342653] Key type .llcrypt registered [ 2574.434705] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_hostid [ 2588.958243] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 2589.952898] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2589.971292] alg: No test for adler32 (adler32-zlib) [ 2590.984315] Lustre: Lustre: Build Version: 2.17.57_103_g3c4c45f [ 2591.189209] LNet: Added LNI 192.168.203.115@tcp [8/256/0/180] [ 2592.825701] Key type lgssc registered [ 2593.823788] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2633.647841] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2633.664051] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2635.011541] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2635.068729] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2635.169425] Lustre: lustre-MDT0000: new disk, initializing [ 2635.262228] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2635.290191] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2639.966244] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 2651.416421] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2651.840494] Lustre: lustre-OST0000: new disk, initializing [ 2651.853832] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2651.862298] Lustre: Skipped 1 previous similar message [ 2651.882521] Lustre: 117726:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2652.044934] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2658.642554] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 2661.441425] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 2661.449525] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 2661.483199] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 2670.205896] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2670.450273] Lustre: lustre-OST0001: new disk, initializing [ 2670.456995] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2670.465744] Lustre: 118588:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2670.623140] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2676.291890] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 2678.826126] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 2678.837298] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 2678.876659] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 2688.993612] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2688.995457] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2689.011207] Lustre: Skipped 1 previous similar message [ 2689.023555] Lustre: Skipped 1 previous similar message [ 2691.437602] Lustre: server umount lustre-MDT0000 complete [ 2698.223446] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 2703.392860] LustreError: 116912:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789266256 with bad export cookie 6053365084442730224 [ 2703.403211] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2703.493672] Lustre: server umount lustre-OST0000 complete [ 2708.383989] Lustre: server umount lustre-OST0001 complete [ 2721.404284] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 22:24:32 (1789266272) [ 2747.032250] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 22:24:57 (1789266297) [ 2757.727416] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 2786.366677] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 22:25:37 (1789266337) [ 2829.368973] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_hostid [ 2839.457604] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 2876.664441] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2876.917179] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2876.953665] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2877.070887] Lustre: lustre-MDT0000: new disk, initializing [ 2877.119584] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2877.133602] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2881.692442] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 2892.945992] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2893.095583] Lustre: 126345:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2893.167862] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2893.170710] Lustre: Skipped 1 previous similar message [ 2893.222334] Lustre: lustre-MDT0001: new disk, initializing [ 2893.303233] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2893.341162] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2893.383679] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2898.304479] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 2908.827756] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2915.832888] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2923.183202] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2923.531258] Lustre: lustre-OST0000: new disk, initializing [ 2923.541747] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2923.548668] Lustre: 128315:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2923.675660] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2930.580550] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 2932.289636] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2932.296505] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2932.442951] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2940.684341] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2945.949916] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2946.147663] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2950.689473] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2950.939340] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2953.195984] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2953.210763] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2953.215131] Lustre: Skipped 1 previous similar message [ 2953.217586] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2953.222582] Lustre: Skipped 1 previous similar message [ 2958.317828] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2958.325615] Lustre: Skipped 1 previous similar message [ 2958.904302] Lustre: server umount lustre-OST0000 complete [ 2963.428593] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2963.448534] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2963.457459] Lustre: Skipped 1 previous similar message [ 2968.608263] Lustre: server umount lustre-MDT0000 complete [ 2972.372401] LustreError: 128313:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789266525 with bad export cookie 6053365084442731449 [ 2972.375132] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2972.381169] LustreError: 128313:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2972.803880] Lustre: server umount lustre-MDT0001 complete [ 2981.719891] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 22:28:53 (1789266533) [ 2987.976203] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2988.434025] LustreError: 130728:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2988.453910] LustreError: 130728:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2988.526201] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2992.441915] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 2993.638255] LustreError: 130729:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2997.732716] LustreError: 130728:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2999.688628] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3004.295295] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 3012.131761] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3018.287956] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3025.428359] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3025.801670] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3025.810898] Lustre: Skipped 1 previous similar message [ 3032.290330] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 3040.173225] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3048.372620] Lustre: *** cfs_fail_loc=159, val=0*** [ 3064.915509] Lustre: lustre-MDT0000: Client cf7a314e-0275-41c6-822c-7156c559fbef (at 192.168.203.15@tcp) reconnecting [ 3066.605710] Lustre: *** cfs_fail_loc=159, val=0*** [ 3066.607589] Lustre: Skipped 5 previous similar messages [ 3082.343359] Lustre: lustre-MDT0000: Client cf7a314e-0275-41c6-822c-7156c559fbef (at 192.168.203.15@tcp) reconnecting [ 3101.667488] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3101.670117] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3101.684938] LustreError: Skipped 1 previous similar message [ 3101.687285] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3101.727802] Lustre: Skipped 2 previous similar messages [ 3106.339091] Lustre: server umount lustre-OST0000 complete [ 3111.915758] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3111.917661] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3111.945631] Lustre: Skipped 1 previous similar message [ 3111.967154] Lustre: Skipped 2 previous similar messages [ 3124.192197] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3124.493617] Lustre: server umount lustre-MDT0000 complete [ 3127.280484] LustreError: 133395:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3127.299736] LustreError: 133395:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3129.004959] LustreError: 130709:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789266682 with bad export cookie 6053365084442732331 [ 3129.010695] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3129.020636] LustreError: 130709:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3129.644846] Lustre: server umount lustre-MDT0001 complete [ 3144.224710] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing unload_modules_local [ 3147.431335] Key type lgssc unregistered [ 3147.814686] LNet: 134509:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3147.826128] LNetError: 134509:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3147.849456] LNet: Removed LNI 192.168.203.115@tcp [ 3149.092414] Key type .llcrypt unregistered [ 3149.098393] Key type ._llcrypt unregistered [ 3175.769546] Key type ._llcrypt registered [ 3175.773168] Key type .llcrypt registered [ 3175.874707] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 3176.579813] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3176.651093] alg: No test for adler32 (adler32-zlib) [ 3177.669648] Lustre: Lustre: Build Version: 2.17.57_103_g3c4c45f [ 3177.961275] LNet: Added LNI 192.168.203.115@tcp [8/256/0/180] [ 3179.672239] Key type lgssc registered [ 3180.903476] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3187.985391] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3188.000638] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3189.361505] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3193.565337] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 3194.849155] LustreError: 135922:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3194.872906] LustreError: 135922:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3198.949432] LustreError: 135923:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3203.560247] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3203.838868] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3208.369306] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 3218.727858] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3226.991853] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3235.456376] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3235.981146] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3243.277801] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 3252.218687] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3259.594456] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3271.100333] Lustre: *** cfs_fail_loc=159, val=0*** [ 3287.636052] Lustre: lustre-MDT0000: Client 13e64631-8aaa-4ac8-965e-21287337bec3 (at 192.168.203.15@tcp) reconnecting [ 3289.822520] Lustre: *** cfs_fail_loc=159, val=0*** [ 3306.032852] Lustre: lustre-MDT0001: Client 13e64631-8aaa-4ac8-965e-21287337bec3 (at 192.168.203.15@tcp) reconnecting [ 3307.490640] Lustre: *** cfs_fail_loc=159, val=0*** [ 3307.505078] Lustre: Skipped 3 previous similar messages [ 3324.030744] Lustre: lustre-MDT0001: Client 13e64631-8aaa-4ac8-965e-21287337bec3 (at 192.168.203.15@tcp) reconnecting [ 3329.398844] Lustre: 138983:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3332.170195] Lustre: *** cfs_fail_loc=159, val=0*** [ 3332.171688] Lustre: Skipped 3 previous similar messages [ 3348.565220] Lustre: lustre-MDT0000: Client 89459d05-2385-4d66-aa21-b3b1c4acda4e (at 192.168.203.15@tcp) reconnecting [ 3350.925667] Lustre: *** cfs_fail_loc=159, val=0*** [ 3350.935357] Lustre: Skipped 14 previous similar messages [ 3366.495204] Lustre: lustre-MDT0000: Client 89459d05-2385-4d66-aa21-b3b1c4acda4e (at 192.168.203.15@tcp) reconnecting [ 3369.212109] Lustre: 139275:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3375.085562] LustreError: lustre-OST0000-osc-MDT0001: operation ost_destroy to node 0@lo failed: rc = -107 [ 3375.089389] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3375.113981] LustreError: Skipped 1 previous similar message [ 3375.169428] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3377.635644] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3377.637100] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3377.679118] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3377.689414] LustreError: Skipped 6 previous similar messages [ 3381.090540] Lustre: server umount lustre-OST0000 complete [ 3386.898987] Lustre: server umount lustre-MDT0000 complete [ 3387.882726] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3387.907133] LustreError: 135918:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3387.944673] LustreError: 135918:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3391.616595] LustreError: 135904:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789266944 with bad export cookie 10124998627803788083 [ 3391.626185] LustreError: 135904:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3391.626476] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3391.915456] Lustre: server umount lustre-MDT0001 complete [ 3406.510683] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing unload_modules_local [ 3410.360342] Key type lgssc unregistered [ 3410.715469] LNet: 140329:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3410.729633] LNetError: 140329:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3410.750570] LNet: Removed LNI 192.168.203.115@tcp [ 3411.672163] Key type .llcrypt unregistered [ 3411.675476] Key type ._llcrypt unregistered [ 3442.208127] Key type ._llcrypt registered [ 3442.211501] Key type .llcrypt registered [ 3442.405617] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 3443.788326] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3443.952256] alg: No test for adler32 (adler32-zlib) [ 3445.135551] Lustre: Lustre: Build Version: 2.17.57_103_g3c4c45f [ 3445.448122] LNet: Added LNI 192.168.203.115@tcp [8/256/0/180] [ 3447.208363] Key type lgssc registered [ 3448.654541] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3457.810754] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3457.827506] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3459.360359] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3464.940433] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 3468.774934] LustreError: 141743:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3468.808310] LustreError: 141743:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3473.889625] LustreError: 141744:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3474.266586] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3474.650530] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3479.423458] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 3487.688638] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3493.727217] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3501.022498] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3501.450349] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3504.597893] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 3508.482139] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 3517.192621] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3518.964153] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 3526.162973] Lustre: 144324:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3529.090243] Lustre: 144372:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3530.678655] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 3531.591256] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 3535.928315] Lustre: 144520:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 3537.113765] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 3538.107221] Lustre: 144629:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3544.546179] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3544.550236] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3544.567773] Lustre: Skipped 1 previous similar message [ 3544.603211] Lustre: Skipped 1 previous similar message [ 3548.920508] Lustre: server umount lustre-OST0000 complete [ 3554.793596] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3554.813602] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3558.998602] Lustre: server umount lustre-MDT0000 complete [ 3559.912094] LustreError: 142751:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3559.939454] LustreError: 142751:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3563.323873] LustreError: 141723:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789267116 with bad export cookie 7957952666492157961 [ 3563.333382] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3563.340907] LustreError: 141723:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3563.798574] Lustre: server umount lustre-MDT0001 complete [ 3575.546988] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing unload_modules_local [ 3579.628027] Key type lgssc unregistered [ 3579.960681] LNet: 145679:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3579.974592] LNetError: 145679:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3579.996817] LNet: Removed LNI 192.168.203.115@tcp [ 3581.052160] Key type .llcrypt unregistered [ 3581.055402] Key type ._llcrypt unregistered [ 3606.566334] Key type ._llcrypt registered [ 3606.568388] Key type .llcrypt registered [ 3606.654651] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 3607.652976] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3607.680147] alg: No test for adler32 (adler32-zlib) [ 3608.710959] Lustre: Lustre: Build Version: 2.17.57_103_g3c4c45f [ 3609.038931] LNet: Added LNI 192.168.203.115@tcp [8/256/0/180] [ 3610.704189] Key type lgssc registered [ 3611.969189] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3620.376360] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3620.415482] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3621.814837] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3626.792606] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 3632.104684] LustreError: 147090:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3632.128346] LustreError: 147090:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3635.983346] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3636.442579] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3641.371513] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 3648.900915] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3654.922874] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3662.372411] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3662.855350] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3666.994070] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 3670.056571] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 3677.434437] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3681.346709] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 3687.483529] Lustre: *** cfs_fail_loc=159, val=0*** [ 3703.821662] Lustre: lustre-MDT0000: Client c819ab04-702a-4ae1-bd89-d489253a13f6 (at 192.168.203.15@tcp) reconnecting [ 3709.151509] Lustre: server umount lustre-OST0000 complete [ 3712.996511] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3713.006379] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3713.018625] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3717.089463] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3717.091864] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3722.216755] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3722.227988] Lustre: Skipped 2 previous similar messages [ 3723.333524] Lustre: server umount lustre-MDT0000 complete [ 3726.417617] LustreError: 147073:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789267279 with bad export cookie 13439109306778293633 [ 3726.424683] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3726.427606] LustreError: 147073:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3736.218793] Lustre: server umount lustre-MDT0001 complete [ 3750.796863] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing unload_modules_local [ 3754.244206] Key type lgssc unregistered [ 3754.465140] LNet: 150755:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3754.480648] LNetError: 150755:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3754.494852] LNet: Removed LNI 192.168.203.115@tcp [ 3755.508685] Key type .llcrypt unregistered [ 3755.510437] Key type ._llcrypt unregistered [ 3782.098072] Key type ._llcrypt registered [ 3782.100652] Key type .llcrypt registered [ 3782.255029] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 3783.033586] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3783.208704] alg: No test for adler32 (adler32-zlib) [ 3784.296292] Lustre: Lustre: Build Version: 2.17.57_103_g3c4c45f [ 3784.622376] LNet: Added LNI 192.168.203.115@tcp [8/256/0/180] [ 3786.296238] Key type lgssc registered [ 3787.358541] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3794.984191] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3795.006749] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3796.408504] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3800.751951] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 3805.677343] LustreError: 152167:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3805.700985] LustreError: 152167:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3810.180121] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3810.649772] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3815.168895] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 3822.767869] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3828.013952] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3835.122741] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3835.308552] LustreError: 153994:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3835.549037] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3840.705884] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 3840.765846] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 3842.904655] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 3849.924255] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3869.594703] Lustre: lustre-MDT0001: evicting 428993fb-f1af-4e9f-8aa7-df1aa8e48c43 (at 192.168.203.15@tcp) by administrative request [ 3869.601202] Lustre: Skipped 1 previous similar message [ 3902.439325] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3902.440561] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3902.472650] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3902.478140] Lustre: Skipped 1 previous similar message [ 3904.355654] Lustre: server umount lustre-OST0000 complete [ 3912.677331] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3912.682109] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3912.691415] Lustre: Skipped 2 previous similar messages [ 3912.701269] Lustre: Skipped 1 previous similar message [ 3914.540259] Lustre: server umount lustre-MDT0000 complete [ 3917.798108] LustreError: 152163:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3917.826236] LustreError: 152163:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3918.436403] LustreError: 152907:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789267471 with bad export cookie 2178874807391837388 [ 3918.442375] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3918.447803] LustreError: 152907:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3918.735363] Lustre: server umount lustre-MDT0001 complete [ 3931.541528] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing unload_modules_local [ 3934.752961] Key type lgssc unregistered [ 3935.132561] LNet: 156234:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3935.140452] LNetError: 156234:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3935.164695] LNet: Removed LNI 192.168.203.115@tcp [ 3936.144715] Key type .llcrypt unregistered [ 3936.148864] Key type ._llcrypt unregistered [ 4004.911965] Key type ._llcrypt registered [ 4004.915941] Key type .llcrypt registered [ 4005.029583] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 4005.658351] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4005.675096] alg: No test for adler32 (adler32-zlib) [ 4006.831638] Lustre: Lustre: Build Version: 2.17.57_103_g3c4c45f [ 4007.054332] LNet: Added LNI 192.168.203.115@tcp [8/256/0/180] [ 4008.712182] Key type lgssc registered [ 4009.737967] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4016.391786] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4016.417156] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4017.845762] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4022.108776] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 4028.397845] LustreError: 158715:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4028.426637] LustreError: 158715:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4030.462820] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4030.953140] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4035.409680] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 4042.381260] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4047.850322] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4054.089928] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4054.473272] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4056.606292] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4061.755873] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 4062.715550] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4069.736440] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4082.145283] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4082.153049] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4082.171774] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4083.176609] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4083.176877] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4083.186455] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4084.846684] Lustre: server umount lustre-OST0000 complete [ 4088.289390] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4088.291841] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4088.304027] Lustre: Skipped 1 previous similar message [ 4088.320051] Lustre: Skipped 2 previous similar messages [ 4093.409708] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4093.414880] Lustre: Skipped 1 previous similar message [ 4093.856660] Lustre: server umount lustre-MDT0000 complete [ 4097.646618] LustreError: 158696:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789267650 with bad export cookie 18107366991319765562 [ 4097.660935] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4097.662342] LustreError: 158696:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4097.916508] Lustre: server umount lustre-MDT0001 complete [ 4108.717996] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing unload_modules_local [ 4111.578058] Key type lgssc unregistered [ 4111.937122] LNet: 162256:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4111.954905] LNetError: 162256:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4111.970185] LNet: Removed LNI 192.168.203.115@tcp [ 4113.055589] Key type .llcrypt unregistered [ 4113.058859] Key type ._llcrypt unregistered [ 4139.385220] Key type ._llcrypt registered [ 4139.387200] Key type .llcrypt registered [ 4139.508421] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 4140.456675] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4140.516309] alg: No test for adler32 (adler32-zlib) [ 4141.579517] Lustre: Lustre: Build Version: 2.17.57_103_g3c4c45f [ 4141.800060] LNet: Added LNI 192.168.203.115@tcp [8/256/0/180] [ 4143.441080] Key type lgssc registered [ 4144.448619] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4160.388887] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 22:48:32 (1789267712) [ 4178.170493] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_hostid [ 4188.487830] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 4224.007396] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4224.029144] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4225.294582] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4225.377965] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4225.495262] Lustre: lustre-MDT0000: new disk, initializing [ 4225.597181] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4225.639846] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4230.081975] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 4240.717357] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4240.883986] Lustre: 166517:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4240.965324] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4240.971965] Lustre: Skipped 1 previous similar message [ 4241.072672] Lustre: lustre-MDT0001: new disk, initializing [ 4241.177266] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4241.212910] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4241.230313] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4245.869955] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 4255.477236] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4260.707509] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4267.112228] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4267.382173] Lustre: lustre-OST0000: new disk, initializing [ 4267.387814] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4267.396641] Lustre: 168485:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4267.503981] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4273.197852] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 4274.772575] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4274.789174] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4274.928215] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4281.411825] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4290.531034] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4290.535471] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4290.537931] LustreError: Skipped 1 previous similar message [ 4290.557254] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4295.659030] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4295.666310] Lustre: Skipped 2 previous similar messages [ 4296.701767] Lustre: server umount lustre-OST0000 complete [ 4300.769589] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4300.771055] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4300.782486] Lustre: Skipped 2 previous similar messages [ 4305.894527] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4305.906710] Lustre: Skipped 2 previous similar messages [ 4306.525988] Lustre: server umount lustre-MDT0000 complete [ 4310.228991] LustreError: 166508:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789267863 with bad export cookie 13362803776639145432 [ 4310.234650] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4310.254068] LustreError: 166508:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4310.701322] Lustre: server umount lustre-MDT0001 complete [ 4319.663733] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing unload_modules_local [ 4321.596336] Key type lgssc unregistered [ 4321.860953] LNet: 170389:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4321.873533] LNetError: 170389:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4321.904741] LNet: Removed LNI 192.168.203.115@tcp [ 4322.703886] Key type .llcrypt unregistered [ 4322.709132] Key type ._llcrypt unregistered [ 4348.953034] Key type ._llcrypt registered [ 4348.960877] Key type .llcrypt registered [ 4349.107392] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 4350.141824] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4350.297863] alg: No test for adler32 (adler32-zlib) [ 4351.425122] Lustre: Lustre: Build Version: 2.17.57_103_g3c4c45f [ 4351.661026] LNet: Added LNI 192.168.203.115@tcp [8/256/0/180] [ 4353.344942] Key type lgssc registered [ 4354.203808] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4363.513131] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4363.549596] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4365.009319] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4369.945259] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 4375.533101] LustreError: 171800:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4375.566499] LustreError: 171800:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4379.818101] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4380.105409] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4384.903991] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 4392.554666] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4397.930321] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4404.956645] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4405.382552] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4410.511870] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4412.743626] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 4422.070690] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4429.891701] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4429.919455] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4429.935756] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4429.958949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4429.995849] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4431.019153] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4431.038981] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4431.060237] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4431.073707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4431.091545] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4432.295083] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4432.316482] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4432.335659] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4432.339959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4432.356650] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4433.293651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4434.305386] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4434.318564] Lustre: Skipped 1 previous similar message [ 4434.339158] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4434.347355] Lustre: Skipped 1 previous similar message [ 4434.353353] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4434.360947] Lustre: Skipped 1 previous similar message [ 4434.365035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4434.374629] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4434.399781] Lustre: Skipped 1 previous similar message [ 4435.247840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4436.206249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4437.157394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4438.095590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4439.093529] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4439.098745] Lustre: Skipped 4 previous similar messages [ 4439.118317] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4439.137042] Lustre: Skipped 4 previous similar messages [ 4439.149252] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4439.152631] Lustre: Skipped 4 previous similar messages [ 4439.171579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4439.189192] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4439.192429] Lustre: Skipped 4 previous similar messages [ 4440.094195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4441.063224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4442.143058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4443.173219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4444.240378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4445.207734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4445.995483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4446.995264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4447.884960] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4447.891165] Lustre: Skipped 8 previous similar messages [ 4447.901898] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4447.912435] Lustre: Skipped 8 previous similar messages [ 4447.915570] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4447.919545] Lustre: Skipped 8 previous similar messages [ 4447.923528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4447.933744] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4447.939430] Lustre: Skipped 8 previous similar messages [ 4448.799274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4449.606062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4450.590951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4451.445358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4452.385644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4453.229497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4454.205548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4455.105205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4456.181588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4457.093142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4458.053199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4459.095767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4460.072316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4460.970620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4461.856237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4462.744038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4463.689474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4464.617653] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4464.627094] Lustre: Skipped 17 previous similar messages [ 4464.640672] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4464.648811] Lustre: Skipped 17 previous similar messages [ 4464.653680] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4464.657701] Lustre: Skipped 17 previous similar messages [ 4464.662523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4464.671890] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4464.674884] Lustre: Skipped 17 previous similar messages [ 4465.534991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4466.331515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4467.156568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4468.085164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4469.157504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4470.229318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4471.210939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4472.225802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4473.423770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4474.397083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4475.377259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4476.451513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4477.432257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4478.389123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4479.318116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4480.251454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4481.152262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4482.028763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4482.841676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4483.834067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4484.769995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4485.789317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4486.804029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4487.990079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4488.838893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4489.715768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4490.730993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4491.694516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4492.697286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4493.556426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4494.416168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4495.275353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4496.247548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4497.253392] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4497.259977] Lustre: Skipped 33 previous similar messages [ 4497.277642] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4497.287681] Lustre: Skipped 33 previous similar messages [ 4497.292591] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4497.297872] Lustre: Skipped 33 previous similar messages [ 4497.303069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4497.311539] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4497.314597] Lustre: Skipped 33 previous similar messages [ 4498.236627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4499.167088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4500.068498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4500.952482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4501.799585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4502.679835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4503.727287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4504.834987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4505.967329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4506.982918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4507.978103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4509.037249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4510.037070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4511.235597] LustreError: 171819:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4511.246366] LustreError: 171819:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4511.261840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4512.091723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4513.240422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4514.304386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4515.202980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4516.225425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4517.129710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4518.098083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4519.068480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4520.078813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4521.070990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4522.038607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4523.078630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4524.144586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4525.215826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4526.291690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4527.264779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4528.143105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4529.118760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4530.043592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4531.079949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4532.081630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4532.926503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4533.908270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4535.033331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4536.018960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4537.197986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4538.115935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4539.133894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4540.018945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4541.123933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4542.010293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4543.373561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4544.852099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4546.157958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4547.205723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4548.194378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4549.078777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4550.063824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4551.370344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4552.630414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4553.789254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4555.189379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4556.289936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4557.625663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4558.899544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4559.977365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4560.690423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4561.811800] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4561.829130] Lustre: Skipped 61 previous similar messages [ 4561.854680] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4561.879667] Lustre: Skipped 61 previous similar messages [ 4561.894097] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4561.911912] Lustre: Skipped 61 previous similar messages [ 4561.923513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4561.950246] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4561.967171] Lustre: Skipped 61 previous similar messages [ 4563.302591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4564.284781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4565.273173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4566.203810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4567.224955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4568.339508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4569.528927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4570.757796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4571.786345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4572.831627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4573.838318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4574.724110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4575.616243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4576.587321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4577.599391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4578.554833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4579.508282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4580.446397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4581.404019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4582.299122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4583.402484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4584.323993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4585.350307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4586.328987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4587.331617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4588.173501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4589.112169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4589.985761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4590.993731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4592.207806] LustreError: 171819:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4592.220137] LustreError: 171819:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4592.229897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4593.204130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4594.107530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4595.004061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4596.106853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4597.074677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4598.163676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4599.294511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4600.413995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4601.473663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4602.533209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4603.844888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4604.808606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4605.822827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4606.847941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4608.099567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4609.202564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4610.084200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4611.274468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4612.314975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4613.345576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4614.365991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4615.356536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4616.250697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4617.048348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4617.930837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4618.950611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4619.746411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4620.584629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4621.438862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4622.289402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4623.149084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4623.848480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4624.632453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4625.411955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4626.152262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4626.981694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4627.755536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4628.443445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4629.196845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4629.955039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4630.801851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4631.521862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4632.320450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4633.096705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4633.945297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4634.763074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4635.572604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4636.485926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4637.419588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4638.203672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4639.003570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4639.755554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4640.567617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4641.329683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4642.252439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4642.991993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4643.733543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4644.586295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4645.456385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4646.239172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4647.085614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4647.844581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4648.658127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4649.403151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4650.463671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4651.264154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4652.212891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4653.121711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4653.968329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4654.729971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4655.629351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4656.369617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4657.261821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4657.928683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4658.714380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4659.586068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4660.467108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4661.293126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4662.206017] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4662.928624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4663.745180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4664.741230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4665.881646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4666.948796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4667.930835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4669.210558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4670.378620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4671.439512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4672.428019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4673.466475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4674.495673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4675.519388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4676.297136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4677.020827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4677.937746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4679.070847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4680.285108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4681.466519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4682.518957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4683.545751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4684.406270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4685.401912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4686.496913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4687.585849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4688.437290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4689.525734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4690.504533] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4690.519328] Lustre: Skipped 136 previous similar messages [ 4690.544739] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4690.566940] Lustre: Skipped 136 previous similar messages [ 4690.574525] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4690.578461] Lustre: Skipped 136 previous similar messages [ 4690.587837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4690.603628] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4690.605652] Lustre: Skipped 136 previous similar messages [ 4691.466514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4692.388779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4693.257147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4694.155266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4695.132548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4696.095670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4697.080417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4697.924269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4698.837252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4699.511771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4700.403786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4701.281261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4702.081255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4702.912497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4703.781676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4704.503248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4705.525853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4706.450864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4707.359385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4708.150289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4709.109196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4710.030246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4710.979910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4712.023234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4712.913368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4713.919857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4714.937339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4716.009035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4717.063538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4718.034939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4718.986954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4720.053781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4720.992605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4721.799058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4722.509570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4723.351239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4724.232969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4724.932375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4725.847396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4726.739315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4727.545567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4728.326359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4729.123925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4730.050662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4730.950084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4731.798388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4732.736066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4733.627251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4734.573377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4735.418446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4736.433985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4737.307728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4738.279623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4739.105405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4740.104446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4740.906741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4741.717287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4742.667412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4743.559168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4744.392926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4745.554413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4746.460473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4747.310796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4748.055889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4748.951531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4749.877026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4750.703658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4751.658309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4752.630668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4753.550432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4754.546475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4755.536483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4756.469165] LustreError: 171819:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -108 [ 4756.479722] LustreError: 171819:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -108 [ 4756.484194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4757.390615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4758.194715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4759.029573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4759.974455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4760.874370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4761.819566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4762.704935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4763.761497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4764.661184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4765.605839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4766.317893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4767.184913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4768.000266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4768.871657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4769.965902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4770.782891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4771.956440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4772.947795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4773.844467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4774.756791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4775.798039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4776.607276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4777.553766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4778.641813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4779.716409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4780.767234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4781.837756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4782.915145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4783.991788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4784.956081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4785.999241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4786.858449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4787.820288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4788.728816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4789.655505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4790.572821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4791.639265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4792.573094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4793.527084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4794.435417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4795.308316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4796.263687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4797.154340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4798.239108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4799.121669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4799.975417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4800.918649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4801.919441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4802.855457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4803.797790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4804.795446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4805.713806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4806.586574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4807.420294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4808.312348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4809.179358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4810.134627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4810.995860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4811.911466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4812.643483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4813.533534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4814.376629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4815.355188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4816.229779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4817.042174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4817.830214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4818.707964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4819.708110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4820.650497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4821.642560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4822.791606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4823.691907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4824.713726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4825.857356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4826.959305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4827.790227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4828.709413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4829.626260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4830.489316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4831.284457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4832.288653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4833.236187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4834.372624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4835.385420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4836.481503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4837.527760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4838.690664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4839.718748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4840.753811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4841.799605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4842.876681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4843.796077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4844.674140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4845.560241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4846.504362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4847.516246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4848.522718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4849.656096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4850.844945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4851.922052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4852.886146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4853.899905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4854.870219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4855.679375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4856.441392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4857.317941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4858.146391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4859.354286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4860.178467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4860.994477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4861.693602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4862.492657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4863.188671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4863.948444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4864.630126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4865.722186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4866.448020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4867.066199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4867.689834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4868.474687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4869.082925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4869.810696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4870.577068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4871.457816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4872.231039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4873.235976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4874.317388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4875.459231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4876.705949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4877.848189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4878.969273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4880.036861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4881.181563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4882.128267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4883.035798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4883.922947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4885.026793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4885.986199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4886.973846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4888.172499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4889.528487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4890.462163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4891.488026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4892.750097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4893.396543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4894.220835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4895.112413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4895.834036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4896.723697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4897.641359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4898.690761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4899.845585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4900.827907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4901.875796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4902.815605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4903.894626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4904.811765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4905.673992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4906.605599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4907.661995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4908.682229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4909.733209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4910.532566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4911.368113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4912.346645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4913.196531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4914.076247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4914.944692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4915.830554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4916.574686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4917.418390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4918.219852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4919.255552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4920.025720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4920.904859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4921.863559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4922.774352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4923.637855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4924.580795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4925.570172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4926.459331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4927.285774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4928.037703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4928.804233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4929.709934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4930.530235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4931.417091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4932.261901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4933.133425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4933.928721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4934.699535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4935.388540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4935.933589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4936.808153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4937.740846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4938.721396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4939.717228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4940.729939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4941.978796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4942.880940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4943.915656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4944.986673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4945.936376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4947.031801] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4947.035389] Lustre: Skipped 276 previous similar messages [ 4947.048331] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4947.054166] Lustre: Skipped 276 previous similar messages [ 4947.057237] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4947.060361] Lustre: Skipped 276 previous similar messages [ 4947.062870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4947.070810] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4947.072897] Lustre: Skipped 276 previous similar messages [ 4948.328687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4949.424819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4950.612400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4951.437941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4952.354583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4953.660681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4954.897018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4955.965105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4956.616457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4957.715031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4958.763164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4959.738858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4960.935299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4962.025158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4963.048878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4964.047475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4964.864326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4965.904765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4966.706941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4967.615733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4968.346642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4969.500554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4970.278454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4971.572274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4972.543531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4973.669985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4974.491840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4975.428694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4976.255044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4977.339398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4978.089397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4979.009638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4979.851586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4980.710429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4981.629567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4982.479491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4983.409521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4984.402696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4985.370069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4986.269543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4987.383986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4988.383845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4989.389526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4990.351926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4991.397775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4992.319390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4993.373751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4994.294741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4995.242535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4996.272269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.306211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4998.058810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4999.043307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5000.042301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.062450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5002.055644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5003.022159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5003.934335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.043566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.915190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5006.945162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.785174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5008.837914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.757663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5010.555214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.377090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5012.321874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.123920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.988841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5014.939753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5015.918518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.028901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.899758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5018.931653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5019.855353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.743639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5021.685815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.544473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5023.526889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.593742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5025.621734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5026.502537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.599491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5028.376551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.379757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5030.419594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5031.350929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5032.277064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.043948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.846810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.656483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.729849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.546951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5037.382704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.026883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.802183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5039.598623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.379690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.154198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.032150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.853744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.551162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.489421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.415164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.345076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.209961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.188117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.309246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.200245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.181968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.927187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.805884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.795574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.831464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.692424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.742626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.763309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.936576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.137627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.140757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.219774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.087058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.038654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.169531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.077799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.072901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.800217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.846863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.868180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.846540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.739347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.759721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.742526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.118386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.289095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.276961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.418320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.362083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.311184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.239078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.505389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.462779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.518719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.015245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.466689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.927261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.916205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.172506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.221455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.082286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.976338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.845512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.835469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.215684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.298101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.298384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.266731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.157235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.180604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.982351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.868879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.717643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.634707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.475370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.518383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.382808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.314144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.230667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.299153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.220653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.238385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.283064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.150990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.981336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.849972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.748537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.555603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.535628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.394699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.250953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.061421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.985626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.873376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.790431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.457408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.299141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.012220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.880894] LustreError: 171819:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5127.894598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.894735] LustreError: 171819:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5128.834811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.830290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.643584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.562861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.343315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.298614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.162513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.153837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.988198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.984300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.941120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.901302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.762102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.713563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.558817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.400621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.242926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.227464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.329906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.239419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.257436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.198777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.052932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.988056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.875919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.786196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.665435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.473148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.487744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.529607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.246514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.126671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.882406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.656866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.474971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.351121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.281855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.192199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.017970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.875060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.789912] LustreError: 171819:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5164.809828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.810856] LustreError: 171819:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5165.693719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.779686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.779746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.651369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.533693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.421162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.242916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.112443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.959667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.811679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.753736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.644541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.482705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.364779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.291580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.236683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.104424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.907471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.736212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.547314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.403960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.331849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.309807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.171690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.045107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.067207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.942886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.906910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.729512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.680271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.608232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.465191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.237351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.132650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.024979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.964908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.731985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.635304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.563683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.570198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.459751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.519653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.487286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.655681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.722348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.732213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.846620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.839581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.905504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.003721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.962333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.988676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.032929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.035574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.950245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.144075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.015973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.894231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.732941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.744975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.561526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.652936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.667081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.598056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.503147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.405048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.292417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.316071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.227342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.237972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.217449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.151488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.062188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.909656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.955246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.830155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.723571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.599408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.546078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.403934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.229705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.165264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.993946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.591957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.466137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.268387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.193403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.145236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.970076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.877798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.716611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.654367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.493101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.384552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.373870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.441851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.336582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.175244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.287884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.299979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.289058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.456195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.417146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.598287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.887756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.905193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.969811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.738913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.734604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.758944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.577161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.587428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.142724] LustreError: 171819:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5271.152588] LustreError: 171819:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5271.160719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.431594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.676064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.831308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.168831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.314401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.603568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.526578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.597260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.537372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.783356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.770971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.671267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.658833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.933519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.016473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.077816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.934610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.890236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.887412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.815757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.767304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.649396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.576130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.550393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.555805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.523593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.725499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.972293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.716138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.576899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.528132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.536895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.422336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.371253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.096489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.944326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.769633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.607362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.414301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.293334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.028018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.893346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.895814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.861274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.773686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.963745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.095407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.076975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.174234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.339912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.719882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.825110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.031914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.043044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.168508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.539479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.940506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.409421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.691362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.014054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.807971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.177475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.429133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.166799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.364491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.782495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.550408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.611701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.477307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.343058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.163953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.022164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.801069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.858763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.612773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.428411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.356194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.299603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.579038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.805534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.128411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.123050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.304667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.535722] LustreError: 171819:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5358.546107] LustreError: 171819:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5358.562693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.435273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.621583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.617873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.679970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.157450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.217982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.250618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.546226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.628231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.862655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.888783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.416661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.535693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.357748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.274891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.578412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.572813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.975427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.830352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.908451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.850945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.947828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.847573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.156704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.417820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.772326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.794560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.482463] LustreError: 173621:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 5797.499991] LustreError: 171819:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6513.633338] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6513.640793] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6513.652807] Lustre: Skipped 453 previous similar messages [ 6513.657133] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6514.152651] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6515.483703] Lustre: server umount lustre-OST0000 complete [ 6520.291894] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6520.311976] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6520.318476] Lustre: Skipped 2 previous similar messages [ 6524.385148] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6524.393740] Lustre: Skipped 1 previous similar message [ 6525.771469] Lustre: server umount lustre-MDT0000 complete [ 6529.504743] LustreError: 171799:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6529.521438] LustreError: 171799:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6529.941602] LustreError: 210289:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789270083 with bad export cookie 16666042026046267819 [ 6529.948398] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6529.954501] LustreError: 210289:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6530.447596] Lustre: server umount lustre-MDT0001 complete [ 6542.627496] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing unload_modules_local [ 6546.045762] Key type lgssc unregistered [ 6546.452677] LNet: 227075:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6546.472261] LNetError: 227075:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6546.499363] LNet: Removed LNI 192.168.203.115@tcp [ 6547.491753] Key type .llcrypt unregistered [ 6547.494381] Key type ._llcrypt unregistered [ 6571.976693] Key type ._llcrypt registered [ 6571.980622] Key type .llcrypt registered [ 6572.167769] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 6573.594722] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6573.614408] alg: No test for adler32 (adler32-zlib) [ 6574.874451] Lustre: Lustre: Build Version: 2.17.57_103_g3c4c45f [ 6575.354882] LNet: Added LNI 192.168.203.115@tcp [8/256/0/180] [ 6577.088181] Key type lgssc registered [ 6578.245404] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6587.227504] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6587.254707] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6588.945185] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6594.050403] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 6599.139728] LustreError: 228488:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6599.163304] LustreError: 228488:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 6603.839968] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6604.149880] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6609.346804] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 6618.749855] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6625.844682] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6633.195587] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6633.573579] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6634.627755] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 6640.207527] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 6648.488181] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6653.610507] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6655.792855] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6655.807154] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6655.810814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6655.820554] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6655.823780] LustreError: 228508:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 6659.553430] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6659.562994] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6659.594916] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6660.588105] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6660.606029] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6660.646302] Lustre: Skipped 1 previous similar message [ 6663.883754] Lustre: server umount lustre-OST0000 complete [ 6675.864165] LustreError: 228508:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 6698.622923] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6699.110204] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6701.166344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6701.191702] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6705.939581] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 6711.399196] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6711.420226] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 6714.838601] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6716.489694] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 6722.684453] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff94f747e65000.ost_server_uuid 50 [ 6724.289120] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff94f747e65000.ost_server_uuid in IDLE state after 0 sec [ 6729.705616] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6729.718550] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6729.733405] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6729.736972] Lustre: Skipped 1 previous similar message [ 6732.261435] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6732.262281] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6732.284194] Lustre: Skipped 1 previous similar message [ 6735.615222] Lustre: server umount lustre-OST0000 complete [ 6742.500109] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6742.505118] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6742.511842] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6742.547754] Lustre: Skipped 2 previous similar messages [ 6746.478627] Lustre: server umount lustre-MDT0000 complete [ 6747.624988] LustreError: 231334:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6747.642075] LustreError: 231334:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6750.072762] LustreError: 228470:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789270303 with bad export cookie 11727208969134807559 [ 6750.088713] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6750.396729] Lustre: server umount lustre-MDT0001 complete [ 6763.051709] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing unload_modules_local [ 6765.875269] Key type lgssc unregistered [ 6766.283548] LNet: 233697:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6766.293809] LNetError: 233697:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6766.333927] LNet: Removed LNI 192.168.203.115@tcp [ 6767.342299] Key type .llcrypt unregistered [ 6767.344288] Key type ._llcrypt unregistered [ 6800.781346] Key type ._llcrypt registered [ 6800.784190] Key type .llcrypt registered [ 6800.882523] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 6801.877378] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6801.946756] alg: No test for adler32 (adler32-zlib) [ 6803.119226] Lustre: Lustre: Build Version: 2.17.57_103_g3c4c45f [ 6803.398460] LNet: Added LNI 192.168.203.115@tcp [8/256/0/180] [ 6805.108192] Key type lgssc registered [ 6806.306901] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6815.797877] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6815.812192] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6817.123424] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6821.925461] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 6827.491305] LustreError: 235421:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6827.509712] LustreError: 235421:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 6830.576880] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6830.964993] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6835.883565] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 6838.940940] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 6845.645638] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6846.080324] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6852.653717] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 6861.003942] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6861.196401] Lustre: lustre-OST0001: new disk, initializing [ 6861.199782] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 6861.204927] Lustre: 237736:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6861.297708] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6862.328249] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 6866.820863] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 6871.132126] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 6871.157971] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 6871.276967] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 6878.037491] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6883.180626] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 6896.775052] Lustre: server umount lustre-MDT0000 complete [ 6897.635845] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6897.644578] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6897.661147] LustreError: 236152:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6900.081299] LustreError: 235403:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789270453 with bad export cookie 8411799807203375759 [ 6900.094193] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6900.097437] LustreError: 235403:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6900.193291] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6900.196668] LustreError: 237452:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6900.200333] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6900.207400] Lustre: Skipped 3 previous similar messages [ 6900.229267] LustreError: 237452:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6902.307774] Lustre: server umount lustre-MDT0001 complete [ 6905.951589] Lustre: server umount lustre-OST0000 complete [ 6909.523798] Lustre: server umount lustre-OST0001 complete [ 6928.341558] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 6938.596658] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6939.065539] LustreError: 240836:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6939.103568] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6944.230469] LustreError: 240837:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6944.752680] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 6953.006443] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6953.467720] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6958.596668] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 6961.629668] Lustre: 241978:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6968.794248] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6975.507983] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 6978.417622] LustreError: 242334:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6978.431620] LustreError: 242334:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6984.106297] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6984.381712] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6984.389655] Lustre: Skipped 1 previous similar message [ 6985.414540] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 6985.422718] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49760) [ 6990.924560] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 6999.023476] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7002.742532] Lustre: 243852:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7014.884444] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7014.893706] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7014.909316] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7014.919899] Lustre: Skipped 1 previous similar message [ 7016.432941] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7016.437720] Lustre: Skipped 1 previous similar message [ 7018.948862] Lustre: server umount mylustre-MDT0000 complete [ 7021.540506] LustreError: 240833:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7021.560127] LustreError: 240833:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 7022.720651] LustreError: 240811:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789270576 with bad export cookie 8411799807203378566 [ 7022.733544] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7022.743983] LustreError: 240811:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7023.215763] Lustre: server umount mylustre-MDT0001 complete [ 7037.533988] Lustre: server umount mylustre-OST0000 complete [ 7051.684871] Lustre: server umount mylustre-OST0001 complete [ 7072.116058] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 7081.529129] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7081.938113] LustreError: 246159:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7082.085720] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7086.605343] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 7095.306209] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7100.480466] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 7103.150462] Lustre: 247300:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7110.111822] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7117.038579] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 7126.079823] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7127.432680] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49792) [ 7127.450407] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 7131.990356] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 7138.879429] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7142.373813] Lustre: 249169:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7151.765647] Lustre: server umount tfs-MDT0000 complete [ 7152.097169] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7152.106843] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7152.116965] Lustre: Skipped 3 previous similar messages [ 7152.132133] LustreError: 246159:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7152.146513] LustreError: 246159:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 7155.270926] LustreError: 246135:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789270708 with bad export cookie 8411799807203380456 [ 7155.283021] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7155.292028] LustreError: 246135:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7155.643485] Lustre: server umount tfs-MDT0001 complete [ 7169.819747] Lustre: server umount tfs-OST0000 complete [ 7184.162247] Lustre: server umount tfs-OST0001 complete [ 7214.635507] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 7225.922981] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7226.480123] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7226.488080] Lustre: Skipped 3 previous similar messages [ 7231.073605] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 7239.931743] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7244.565804] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 7247.420674] Lustre: 252566:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7254.391855] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7261.299263] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 7270.252823] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7271.547390] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49824) [ 7271.569581] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 7277.216544] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 7285.262757] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7289.785598] Lustre: 254437:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7298.305107] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 23:40:49 (1789270849) [ 7317.476163] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7317.488731] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7317.506347] Lustre: Skipped 3 previous similar messages [ 7317.514555] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7321.982859] Lustre: server umount lustre-MDT0000 complete [ 7323.130832] LustreError: 251423:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7323.168289] LustreError: 251423:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 11 previous similar messages [ 7326.116301] LustreError: 251399:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789270879 with bad export cookie 8411799807203382122 [ 7326.117644] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7326.126263] LustreError: 251399:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7328.235758] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7328.262191] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7328.264498] Lustre: Skipped 4 previous similar messages [ 7328.287234] Lustre: Skipped 4 previous similar messages [ 7332.717061] Lustre: server umount lustre-MDT0001 complete [ 7351.264125] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7351.585380] Lustre: server umount lustre-OST0000 complete [ 7370.208351] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7370.483350] Lustre: server umount lustre-OST0001 complete [ 7376.983035] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_hostid [ 7388.331828] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 7426.109216] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7426.282311] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7426.301305] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7426.352604] Lustre: lustre-MDT0000: new disk, initializing [ 7426.402055] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 7426.406861] Lustre: Skipped 3 previous similar messages [ 7426.433061] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7433.117351] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 7447.882492] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7447.973681] Lustre: 257626:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7448.035668] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7448.038881] Lustre: Skipped 1 previous similar message [ 7448.106434] Lustre: lustre-MDT0001: new disk, initializing [ 7448.210067] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7448.230406] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7454.801147] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 7466.607118] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7466.943414] Lustre: lustre-OST0000: new disk, initializing [ 7466.950136] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7466.962323] Lustre: 259211:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7468.415624] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7468.433835] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7468.578150] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7473.797488] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 7484.744793] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7484.945479] Lustre: lustre-OST0001: new disk, initializing [ 7484.951981] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7484.968258] Lustre: 260081:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7486.473569] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7486.481845] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7486.631651] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7492.314305] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 7505.278222] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 23:44:16 (1789271056) [ 7523.111866] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 23:44:34 (1789271074) [ 7540.199787] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7540.201376] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7540.214527] LustreError: Skipped 1 previous similar message [ 7540.218170] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7540.232307] Lustre: Skipped 1 previous similar message [ 7540.235773] Lustre: Skipped 2 previous similar messages [ 7544.314949] Lustre: server umount lustre-OST0000 complete [ 7558.119099] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7558.129476] Lustre: Skipped 11 previous similar messages [ 7562.208595] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7562.419178] Lustre: server umount lustre-MDT0000 complete [ 7566.328977] LustreError: 257617:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789271119 with bad export cookie 8411799807203383676 [ 7566.335262] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7566.343946] LustreError: 257617:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7572.913922] Lustre: server umount lustre-MDT0001 complete [ 7584.658809] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing unload_modules_local [ 7657.650383] Lustre: server umount lustre-OST0001 complete [ 7664.372797] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_hostid [ 7679.521159] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 7719.952088] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7720.175276] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7720.196142] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7720.239217] Lustre: lustre-MDT0000: new disk, initializing [ 7720.288725] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7720.294859] Lustre: Skipped 3 previous similar messages [ 7720.304501] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7724.435480] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 7734.359602] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7734.496730] Lustre: 264942:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7734.768202] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7738.857483] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 7747.683860] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7753.093762] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7759.335312] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7759.563209] Lustre: lustre-OST0000: new disk, initializing [ 7759.565615] Lustre: Skipped 1 previous similar message [ 7759.568904] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7759.571899] Lustre: Skipped 2 previous similar messages [ 7759.576367] Lustre: 266908:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7761.195921] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7761.208860] Lustre: Skipped 1 previous similar message [ 7761.223238] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7761.371749] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7765.200665] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 7773.851980] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7781.862848] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7781.863096] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7781.863146] Lustre: Skipped 4 previous similar messages [ 7781.889899] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7781.933473] Lustre: Skipped 3 previous similar messages [ 7786.192724] Lustre: server umount lustre-OST0000 complete [ 7804.385422] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7804.543743] Lustre: server umount lustre-MDT0000 complete [ 7807.463770] LustreError: 264952:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7807.481107] LustreError: 264952:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 14 previous similar messages [ 7808.038624] LustreError: 264934:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789271361 with bad export cookie 8411799807203385139 [ 7808.040828] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7808.050743] LustreError: 264934:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7808.331671] Lustre: server umount lustre-MDT0001 complete [ 7817.822436] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing unload_modules_local [ 7820.228835] Key type lgssc unregistered [ 7820.530378] LNet: 268668:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7820.536772] LNetError: 268668:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7820.551081] LNet: Removed LNI 192.168.203.115@tcp [ 7821.297258] Key type .llcrypt unregistered [ 7821.303534] Key type ._llcrypt unregistered [ 7845.145645] Key type ._llcrypt registered [ 7845.148134] Key type .llcrypt registered [ 7845.252990] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 7846.330355] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7846.387324] alg: No test for adler32 (adler32-zlib) [ 7847.458113] Lustre: Lustre: Build Version: 2.17.57_103_g3c4c45f [ 7847.706693] LNet: Added LNI 192.168.203.115@tcp [8/256/0/180] [ 7849.352165] Key type lgssc registered [ 7850.427728] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7857.343461] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7857.358657] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7858.723398] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7862.785666] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 7868.898374] LustreError: 270128:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7868.930017] LustreError: 270128:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7870.721500] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7871.110796] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7875.293943] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 7882.545768] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7887.915798] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7894.518040] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7894.827792] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7900.429717] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 7904.071569] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 7908.016549] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7908.998923] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 7911.395720] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7911.410094] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7911.439192] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7914.469904] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7914.472615] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7914.477273] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7919.595075] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7919.597768] Lustre: Skipped 1 previous similar message [ 7924.705996] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7924.712126] Lustre: Skipped 2 previous similar messages [ 7925.217180] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7925.460481] Lustre: server umount lustre-OST0000 complete [ 7932.387611] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7932.401441] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7932.420158] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7934.958348] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7936.338393] Lustre: server umount lustre-MDT0000 complete [ 7939.638250] LustreError: 270109:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789271492 with bad export cookie 1024710779271456251 [ 7939.669043] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7940.108842] Lustre: server umount lustre-MDT0001 complete [ 7950.351377] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing unload_modules_local [ 7952.932456] Key type lgssc unregistered [ 7953.189957] LNet: 273707:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7953.197862] LNetError: 273707:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7953.220678] LNet: Removed LNI 192.168.203.115@tcp [ 7954.030091] Key type .llcrypt unregistered [ 7954.032888] Key type ._llcrypt unregistered [ 7968.466258] Key type ._llcrypt registered [ 7968.471894] Key type .llcrypt registered [ 7968.565294] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 7969.208042] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7969.288902] alg: No test for adler32 (adler32-zlib) [ 7970.296478] Lustre: Lustre: Build Version: 2.17.57_103_g3c4c45f [ 7970.467933] LNet: Added LNI 192.168.203.115@tcp [8/256/0/180] [ 7972.104169] Key type lgssc registered [ 7973.099807] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7980.683213] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7980.730187] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7996.256437] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8001.376298] LustreError: 274896:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.115@tcp: failed processing log, type 4: rc = -110 [ 8028.000204] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8034.109468] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 8042.343941] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8044.788644] Lustre: server umount lustre-OST0000 complete [ 8062.110616] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing unload_modules_local [ 8064.905701] Key type lgssc unregistered [ 8065.223577] LNet: 276526:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8065.240719] LNetError: 276526:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8065.258726] LNet: Removed LNI 192.168.203.115@tcp [ 8066.082805] Key type .llcrypt unregistered [ 8066.088848] Key type ._llcrypt unregistered [ 8103.960112] Key type ._llcrypt registered [ 8103.961543] Key type .llcrypt registered [ 8104.092300] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 8105.175547] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8105.206170] alg: No test for adler32 (adler32-zlib) [ 8106.254449] Lustre: Lustre: Build Version: 2.17.57_103_g3c4c45f [ 8106.522535] LNet: Added LNI 192.168.203.115@tcp [8/256/0/180] [ 8108.192338] Key type lgssc registered [ 8109.071891] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8119.727293] loop0: detected capacity change from 0 to 204800000 [ 8120.031037] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8120.042369] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8120.319743] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8121.311652] loop0: detected capacity change from 0 to 204800000 [ 8121.551420] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8121.576022] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8121.963992] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8122.790782] loop0: detected capacity change from 0 to 204800000 [ 8122.927846] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8122.941895] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8123.014258] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8123.757222] loop0: detected capacity change from 0 to 204800000 [ 8123.885866] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8124.695239] loop0: detected capacity change from 0 to 204800000 [ 8124.708949] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8127.787624] loop0: detected capacity change from 0 to 204800000 [ 8127.825754] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8130.549893] loop0: detected capacity change from 0 to 204800000 [ 8130.591031] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8134.029461] loop0: detected capacity change from 0 to 204800000 [ 8134.056609] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8137.756918] loop0: detected capacity change from 0 to 204800000 [ 8137.821113] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8137.851931] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8138.009469] Lustre: 279933:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8138.037878] Lustre: 279933:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8138.059832] Lustre: 279933:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8138.085345] Lustre: 279933:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8138.098655] Lustre: 279933:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8138.114242] Lustre: 279933:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8138.132683] Lustre: 279933:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8138.143590] Lustre: 279933:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8138.154971] Lustre: 279933:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8138.167481] Lustre: 279933:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8138.180706] Lustre: 279933:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8138.191978] Lustre: 279933:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8138.202403] Lustre: 279933:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8138.215441] Lustre: 279933:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8143.544268] Lustre: server umount MGS complete [ 8144.279459] loop0: detected capacity change from 0 to 204800000 [ 8144.393855] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8145.031899] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8145.079572] Lustre: lustre-MDT0000: Aborting client recovery [ 8145.957941] loop1: detected capacity change from 0 to 204800000 [ 8146.021595] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8146.131494] Lustre: 280364:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8146.148532] Lustre: 280364:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8146.156098] Lustre: 280364:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8146.171472] Lustre: 280364:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8146.179771] Lustre: 280364:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8146.199094] Lustre: 280364:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8146.208919] Lustre: 280364:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8146.220393] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8146.440314] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8146.486831] Lustre: lustre-MDT0001: Aborting client recovery [ 8147.337196] loop2: detected capacity change from 0 to 204800000 [ 8147.512239] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8147.560654] Lustre: 280443:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8147.567409] Lustre: 280443:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8147.579829] Lustre: 280443:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8147.590065] Lustre: 280443:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8147.598471] Lustre: 280443:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8147.606835] Lustre: 280443:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8147.613757] Lustre: 280443:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8147.686756] Lustre: Found index 0 for lustre-OST0000, updating log [ 8147.701965] Lustre: 280461:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8147.755805] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8147.773014] Lustre: lustre-OST0000: Aborting recovery [ 8148.690678] loop3: detected capacity change from 0 to 204800000 [ 8148.793624] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8148.915738] Lustre: 280528:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8148.938719] Lustre: 280528:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8148.948128] Lustre: 280528:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8148.953690] Lustre: 280528:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8148.960967] Lustre: 280528:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8148.968240] Lustre: 280528:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8148.975330] Lustre: 280528:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8148.996371] Lustre: Found index 1 for lustre-OST0001, updating log [ 8149.035644] Lustre: 280533:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8149.160379] Lustre: lustre-OST0001: Aborting recovery [ 8154.636225] LustreError: 280308:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 9, retries 0, failed: rc = -108 [ 8155.645848] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8155.658335] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8155.729569] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8155.803185] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8156.668177] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8156.707328] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8158.245576] LustreError: 280275:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1259us: rc = -2 [ 8158.807393] LustreError: 280276:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 579us: rc = -2 [ 8158.819929] LustreError: 280276:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 22 previous similar messages [ 8172.001682] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8172.007299] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8172.008366] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8172.057534] Lustre: Skipped 3 previous similar messages [ 8177.121801] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8177.142889] Lustre: Skipped 6 previous similar messages [ 8182.242031] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8182.249947] Lustre: Skipped 3 previous similar messages [ 8186.337110] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8186.499980] Lustre: server umount lustre-MDT0000 complete [ 8187.310465] LustreError: 280259:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789271740 with bad export cookie 6223211917020350259 [ 8187.331829] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8187.338226] LustreError: 280259:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8187.377032] LustreError: 280765:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8187.402669] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8187.402954] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8187.432224] LustreError: 280765:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8192.481152] LustreError: 280273:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8192.481172] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8192.502594] LustreError: 280273:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8192.519263] Lustre: Skipped 2 previous similar messages [ 8193.769604] Lustre: server umount lustre-MDT0001 complete [ 8200.850894] Lustre: server umount lustre-OST0000 complete [ 8208.107452] Lustre: server umount lustre-OST0001 complete [ 8216.680483] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 23:56:07 (1789271767) [ 8244.682483] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_hostid [ 8255.338219] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 8297.158852] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8297.326407] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8297.365206] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8297.415338] Lustre: lustre-MDT0000: new disk, initializing [ 8297.469437] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8297.472606] Lustre: Skipped 1 previous similar message [ 8297.480237] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8302.357908] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 8312.430464] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8312.568195] Lustre: 284602:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8312.681874] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8312.690575] Lustre: Skipped 1 previous similar message [ 8312.765435] Lustre: lustre-MDT0001: new disk, initializing [ 8312.880635] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8312.926921] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8312.938234] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8316.994698] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 8325.995138] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8331.557218] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8338.065070] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8338.438459] Lustre: lustre-OST0000: new disk, initializing [ 8338.441850] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8338.452873] Lustre: 286572:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8338.543622] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8344.183432] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 8345.714274] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8345.735986] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8345.872058] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8354.231277] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8360.616666] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8373.713847] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8385.088641] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8436.199501] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8436.213399] Lustre: Skipped 1 previous similar message [ 8436.224272] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8436.233312] Lustre: Skipped 1 previous similar message [ 8439.700672] Lustre: server umount lustre-OST0000 complete [ 8445.934474] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8445.942624] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8445.972256] Lustre: Skipped 1 previous similar message [ 8449.547841] Lustre: server umount lustre-MDT0000 complete [ 8451.555607] LustreError: 285974:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8451.575387] LustreError: 285974:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8453.498025] LustreError: 284593:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789272006 with bad export cookie 6223211917020352961 [ 8453.505684] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8453.508077] LustreError: 284593:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8453.776443] Lustre: server umount lustre-MDT0001 complete [ 8464.397476] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8469.065511] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 8473.352851] Lustre: server umount MGS complete [ 8483.955885] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8484.518938] LustreError: 290630:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8484.563511] LustreError: 290643:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8484.730591] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8489.956847] LustreError: 290631:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8491.639253] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 8499.180898] LustreError: 290631:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8499.203967] LustreError: 290631:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8501.942275] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8509.157727] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 8519.255822] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8529.088691] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8539.020807] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8539.507871] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8539.520202] Lustre: Skipped 1 previous similar message [ 8547.275162] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 8554.919590] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8584.680405] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8584.685977] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8584.696187] Lustre: Skipped 1 previous similar message [ 8584.698489] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8584.725247] Lustre: Skipped 4 previous similar messages [ 8590.052731] Lustre: server umount lustre-OST0000 complete [ 8594.916206] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8594.923183] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8594.929745] Lustre: Skipped 1 previous similar message [ 8608.225026] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8608.412412] Lustre: server umount lustre-MDT0000 complete [ 8610.276087] LustreError: 293375:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8610.297770] LustreError: 293375:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8612.594995] LustreError: 290612:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789272165 with bad export cookie 6223211917020354802 [ 8612.600098] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8612.612475] LustreError: 290612:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8613.025223] Lustre: server umount lustre-MDT0001 complete [ 8623.457712] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing unload_modules_local [ 8627.160278] Key type lgssc unregistered [ 8627.615709] LNet: 294593:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8627.625284] LNetError: 294593:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8628.657741] LNet: Removed LNI 192.168.203.115@tcp [ 8630.036190] Key type .llcrypt unregistered [ 8630.037957] Key type ._llcrypt unregistered [ 8668.215368] Key type ._llcrypt registered [ 8668.218122] Key type .llcrypt registered [ 8668.337898] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_hostid [ 8681.665377] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 8682.627239] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8682.641586] alg: No test for adler32 (adler32-zlib) [ 8683.672744] Lustre: Lustre: Build Version: 2.17.57_103_g3c4c45f [ 8683.965639] LNet: Added LNI 192.168.203.115@tcp [8/256/0/180] [ 8685.705395] Key type lgssc registered [ 8687.015542] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8728.137891] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8728.168129] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8729.392358] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8729.440164] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8729.573271] Lustre: lustre-MDT0000: new disk, initializing [ 8729.674943] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8729.709846] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8734.440289] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 8745.141621] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8745.223059] Lustre: 298547:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8745.291339] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8745.297772] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 8745.299030] Lustre: Skipped 1 previous similar message [ 8745.362501] Lustre: lustre-MDT0001: new disk, initializing [ 8745.439062] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8745.484697] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8745.503510] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8750.038314] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 8760.542368] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8766.106886] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8773.208440] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8773.420388] Lustre: lustre-OST0000: new disk, initializing [ 8773.423942] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8773.432507] Lustre: 300516:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8773.492194] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8779.087920] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 8779.820423] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8779.834996] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8779.925306] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8787.753950] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8795.937384] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8809.500251] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8819.586236] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8827.541786] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8877.553645] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8877.558362] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8877.560240] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8877.567862] Lustre: Skipped 1 previous similar message [ 8877.582761] Lustre: Skipped 1 previous similar message [ 8880.620987] Lustre: server umount lustre-OST0000 complete [ 8887.789958] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8887.793754] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8887.808099] Lustre: Skipped 1 previous similar message [ 8892.898071] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8892.908125] Lustre: Skipped 2 previous similar messages [ 8898.022493] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8899.040781] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8899.224361] Lustre: server umount lustre-MDT0000 complete [ 8904.880701] LustreError: 298540:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789272458 with bad export cookie 17242367183935435373 [ 8904.888231] LustreError: 298540:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8904.891130] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8905.213786] Lustre: server umount lustre-MDT0001 complete [ 8915.927800] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8920.788460] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 8926.499792] Lustre: server umount MGS complete [ 8940.103417] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8940.743206] LustreError: 304559:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8940.766269] LustreError: 304559:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8940.784984] LustreError: 304572:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8940.864707] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8944.649554] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 8946.152734] LustreError: 304560:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8951.266539] LustreError: 304559:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8951.995351] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8952.219992] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8956.388861] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 8963.830088] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8971.461958] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8978.733744] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8979.039860] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8985.832700] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 8993.553853] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9018.337228] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9018.345528] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9018.350311] LustreError: Skipped 1 previous similar message [ 9018.352809] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9018.368728] Lustre: Skipped 1 previous similar message [ 9018.401764] Lustre: Skipped 2 previous similar messages [ 9024.509141] Lustre: server umount lustre-OST0000 complete [ 9033.697279] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9036.195769] Lustre: server umount lustre-MDT0000 complete [ 9038.820854] LustreError: 304556:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9038.846698] LustreError: 304556:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9040.665947] LustreError: 304540:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789272594 with bad export cookie 17242367183935437326 [ 9040.682577] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9040.686423] LustreError: 304540:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9041.185388] Lustre: server umount lustre-MDT0001 complete [ 9055.331619] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing unload_modules_local [ 9058.920155] Key type lgssc unregistered [ 9059.321629] LNet: 308505:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9059.330489] LNetError: 308505:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9059.347180] LNet: Removed LNI 192.168.203.115@tcp [ 9060.395890] Key type .llcrypt unregistered [ 9060.404386] Key type ._llcrypt unregistered [ 9082.863657] Key type ._llcrypt registered [ 9082.865494] Key type .llcrypt registered [ 9082.975679] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_hostid [ 9097.483300] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 9098.711806] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9098.778194] alg: No test for adler32 (adler32-zlib) [ 9099.856232] Lustre: Lustre: Build Version: 2.17.57_103_g3c4c45f [ 9100.094633] LNet: Added LNI 192.168.203.115@tcp [8/256/0/180] [ 9101.776415] Key type lgssc registered [ 9102.997772] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9124.595965] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [ 9139.050749] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9139.084028] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9140.375587] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9140.392932] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9140.459145] Lustre: lustre-MDT0000: new disk, initializing [ 9140.541377] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9140.561669] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9145.181559] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [ 9154.949411] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9155.187691] Lustre: lustre-OST0000: new disk, initializing [ 9155.199165] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9155.206454] Lustre: Skipped 1 previous similar message [ 9155.213527] Lustre: 312555:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9155.278687] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9157.973533] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9157.989125] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9158.018520] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9161.029993] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [10182.118649] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10182.145186] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10184.183779] Lustre: server umount lustre-MDT0000 complete [10188.287461] LustreError: 311743:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789273741 with bad export cookie 3049648481779885881 [10188.310500] LustreError: 311743:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10188.311304] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10188.485665] Lustre: server umount lustre-OST0000 complete [10215.775858] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_hostid [10225.526685] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [10263.516284] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10263.694492] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10263.713399] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10263.776827] Lustre: lustre-MDT0000: new disk, initializing [10263.831628] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10263.846990] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10271.007874] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [10286.824772] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10286.991763] Lustre: 316914:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10287.043192] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10287.046813] Lustre: Skipped 1 previous similar message [10287.129785] Lustre: lustre-MDT0001: new disk, initializing [10287.285629] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10287.401502] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10287.428963] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10292.268700] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [10302.892385] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10308.544959] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10315.895657] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10316.195614] Lustre: lustre-OST0000: new disk, initializing [10316.198107] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10316.203763] Lustre: 318883:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10316.308624] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10322.005303] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10322.025254] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10322.190605] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10322.869624] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [10333.690635] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10338.425664] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10338.653194] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10343.343732] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10343.625265] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10348.004341] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10348.006529] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10348.011573] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10348.012708] LustreError: Skipped 1 previous similar message [10348.039760] Lustre: Skipped 1 previous similar message [10351.652996] Lustre: server umount lustre-OST0000 complete [10358.244743] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10358.251759] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10358.264236] Lustre: Skipped 1 previous similar message [10358.281927] Lustre: Skipped 2 previous similar messages [10362.245481] Lustre: server umount lustre-MDT0000 complete [10365.262281] LustreError: 316906:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789273918 with bad export cookie 3049648481782826700 [10365.277817] LustreError: 316906:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10365.279157] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10365.546437] Lustre: server umount lustre-MDT0001 complete [10368.430562] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10370.049623] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 00:32:01 (1789273921) [10376.363195] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10376.844936] LustreError: 321193:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10376.863645] LustreError: 321193:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10376.902668] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10381.334245] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [10382.312533] LustreError: 321194:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10387.428740] LustreError: 321193:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10389.358267] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10389.649184] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10394.968523] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [10406.335298] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10416.208384] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10423.676870] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10423.997681] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10430.444950] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [10438.380790] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10444.857401] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10445.078598] Lustre: lustre-OST0001: new disk, initializing [10445.082231] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10445.097534] Lustre: 324032:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10445.204145] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10451.520710] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [10451.526134] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10451.534690] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10451.581668] LustreError: 324388:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10452.648338] LustreError: 324388:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10452.662599] LustreError: 324388:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10453.674143] LustreError: 324396:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10455.715411] LustreError: 324396:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10455.726087] LustreError: 324396:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [10459.845209] LustreError: 324396:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10459.868487] LustreError: 324396:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [10461.344099] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10461.622560] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10468.078940] LustreError: 324396:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10468.113145] LustreError: 324396:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 16 previous similar messages [10468.957505] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff94f747e62800.ost_server_uuid 50 [10470.858437] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff94f747e62800.ost_server_uuid in FULL state after 0 sec [10477.793704] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff94f747e62800.ost_server_uuid 50 [10479.462044] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff94f747e62800.ost_server_uuid in IDLE state after 0 sec [10481.479411] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10497.014151] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10497.019396] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10497.029389] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10497.041334] Lustre: Skipped 1 previous similar message [10502.148412] Lustre: lustre-OST0001: Not available for connect from 192.168.203.15@tcp (stopping) [10502.163569] Lustre: Skipped 3 previous similar messages [10509.283952] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10509.424990] Lustre: server umount lustre-OST0001 complete [10512.389947] LustreError: 325571:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.203.15@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10518.588808] LustreError: 324626:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10518.654914] LustreError: 324626:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10521.574365] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10521.577116] LustreError: Skipped 1 previous similar message [10521.585756] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10521.611157] Lustre: Skipped 1 previous similar message [10521.626921] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10521.637342] Lustre: Skipped 5 previous similar messages [10528.741623] LustreError: 323018:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10528.758740] LustreError: 323018:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10534.373894] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10534.887936] Lustre: server umount lustre-OST0000 complete [10543.587204] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10543.593720] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10543.603630] Lustre: Skipped 1 previous similar message [10543.609117] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10543.621672] Lustre: Skipped 6 previous similar messages [10546.444908] Lustre: server umount lustre-MDT0000 complete [10549.218263] LustreError: 321194:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10549.233558] LustreError: 321194:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10550.759172] LustreError: 321173:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789274104 with bad export cookie 3049648481782827582 [10550.760032] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10550.770233] LustreError: 321173:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10551.186937] Lustre: server umount lustre-MDT0001 complete [10562.450870] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing unload_modules_local [10565.560876] Key type lgssc unregistered [10565.836941] LNet: 326731:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10565.846698] LNetError: 326731:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10565.873644] LNet: Removed LNI 192.168.203.115@tcp [10566.873168] Key type .llcrypt unregistered [10566.875316] Key type ._llcrypt unregistered [10598.789298] Key type ._llcrypt registered [10598.792543] Key type .llcrypt registered [10598.901944] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [10599.776490] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10599.924114] alg: No test for adler32 (adler32-zlib) [10601.025802] Lustre: Lustre: Build Version: 2.17.57_103_g3c4c45f [10601.349981] LNet: Added LNI 192.168.203.115@tcp [8/256/0/180] [10603.128554] Key type lgssc registered [10604.098309] Lustre: Echo OBD driver; http://www.lustre.org/ [10612.995176] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10613.026869] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10614.572322] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10619.495755] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [10623.981514] LustreError: 328149:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10624.011703] LustreError: 328149:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10629.090697] LustreError: 328150:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10630.625508] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10634.209880] LustreError: 328149:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10639.350411] LustreError: 328150:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10642.411815] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10643.465315] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10650.597746] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [10660.784409] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10661.046630] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10667.324871] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [10669.237140] LustreError: 329831:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10669.267428] LustreError: 329831:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10675.559300] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10679.787409] LustreError: 329832:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10679.808373] LustreError: 329832:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10682.765344] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10682.942571] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10684.985301] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10685.023759] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10689.779972] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [10698.264316] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10706.236915] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10706.587832] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10818.020573] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10818.039493] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10818.067050] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10818.071532] Lustre: Skipped 1 previous similar message [10820.651659] Lustre: server umount lustre-OST0001 complete [10823.141576] LustreError: 330955:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10823.169832] LustreError: 330955:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10833.380971] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10833.384753] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10833.386049] LustreError: Skipped 1 previous similar message [10833.393960] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10833.396447] Lustre: Skipped 2 previous similar messages [10833.434878] Lustre: Skipped 1 previous similar message [10838.503683] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10838.523675] Lustre: Skipped 1 previous similar message [10843.650099] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10843.668338] Lustre: Skipped 1 previous similar message [10844.128404] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10844.448975] Lustre: server umount lustre-OST0000 complete [10853.345393] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10853.358719] LustreError: Skipped 1 previous similar message [10853.369061] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10853.392692] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10856.340121] Lustre: server umount lustre-MDT0000 complete [10858.982331] LustreError: 328150:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10859.015445] LustreError: 328150:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [10860.991332] LustreError: 331818:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789274414 with bad export cookie 10913448424572424473 [10860.999650] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10861.004094] LustreError: 331818:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10861.410734] Lustre: server umount lustre-MDT0001 complete [10876.566217] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing unload_modules_local [10879.875607] Key type lgssc unregistered [10880.324244] LNet: 333102:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10880.336089] LNetError: 333102:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10880.360171] LNet: Removed LNI 192.168.203.115@tcp [10881.512675] Key type .llcrypt unregistered [10881.515576] Key type ._llcrypt unregistered [10927.663950] Key type ._llcrypt registered [10927.666825] Key type .llcrypt registered [10927.749533] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [10928.793620] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10928.808955] alg: No test for adler32 (adler32-zlib) [10929.796919] Lustre: Lustre: Build Version: 2.17.57_103_g3c4c45f [10930.064234] LNet: Added LNI 192.168.203.115@tcp [8/256/0/180] [10931.744167] Key type lgssc registered [10932.366204] Lustre: Echo OBD driver; http://www.lustre.org/ [10940.611226] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10940.623341] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10942.036215] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10947.724955] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [10952.178279] LustreError: 335387:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10952.214754] LustreError: 335387:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10957.298552] LustreError: 335388:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10960.280163] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10960.542283] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10965.730311] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [10968.972186] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [10975.823631] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10976.233828] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10982.203589] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [10985.459817] LustreError: 336878:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10987.562715] LustreError: 337370:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10987.583293] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [10987.584760] LustreError: 337370:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10991.399339] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10991.660505] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10996.744415] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [10997.947526] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [11007.354908] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11019.434379] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11032.034111] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11032.041359] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11032.072260] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11032.549753] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11032.582207] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11032.595166] Lustre: Skipped 2 previous similar messages [11037.665658] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11037.676218] Lustre: Skipped 3 previous similar messages [11038.342707] Lustre: server umount lustre-MDT0000 complete [11042.789285] LustreError: 336391:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11042.832932] LustreError: 336391:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11043.900853] LustreError: 335369:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789274597 with bad export cookie 9786022089584578512 [11043.935058] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11044.519226] Lustre: server umount lustre-MDT0001 complete [11059.990615] Lustre: server umount lustre-OST0000 complete [11063.792649] Lustre: 334652:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789274601/real 1789274601] req@ffff8a07bd655f80 x1876190280352512/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789274617 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11063.813624] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11063.821814] Lustre: Skipped 2 previous similar messages [11064.329298] Lustre: server umount lustre-OST0001 complete [11087.050211] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [11097.902743] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11098.394389] LustreError: 340654:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11098.474620] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11103.301610] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [11112.787897] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11113.294893] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11117.994070] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [11120.559923] Lustre: 341797:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11127.319098] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11134.674531] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [11137.897798] LustreError: 342151:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11137.929952] LustreError: 342151:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11143.044352] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11144.490697] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11144.769196] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11144.775722] Lustre: Skipped 1 previous similar message [11149.846509] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11152.412488] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [11160.122243] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11163.583406] Lustre: 343666:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11169.761724] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11169.783829] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11169.806153] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11173.392234] Lustre: server umount lustre-MDT0000 complete [11175.400249] LustreError: 341387:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11175.430131] LustreError: 341387:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [11178.696823] LustreError: 340636:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789274732 with bad export cookie 9786022089584580206 [11178.706378] LustreError: 340636:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [11178.708557] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11179.570790] Lustre: server umount lustre-MDT0001 complete [11196.320336] Lustre: 334652:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789274733/real 1789274733] req@ffff8a079812bb80 x1876190280400768/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789274749 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11196.361751] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11196.381948] Lustre: Skipped 3 previous similar messages [11196.660491] Lustre: server umount lustre-OST0000 complete [11199.456122] Lustre: 334649:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789274736/real 1789274736] req@ffff8a079812b800 x1876190280401024/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789274752 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11200.553465] Lustre: server umount lustre-OST0001 complete [11204.354391] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11205.958355] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11208.140142] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 00:45:59 (1789274759) [11232.418401] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [11241.482548] loop0: detected capacity change from 0 to 1099511627776 [11242.257145] SGI XFS with ACLs, security attributes, quota, no debug enabled [11242.262501] *** VALIDATE xfs *** [11242.269580] XFS (loop0): Mounting V5 Filesystem [11242.323656] XFS (loop0): Ending clean mount [11244.535953] loop1: detected capacity change from 0 to 18691697672192 [11350.527416] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_hostid [11363.770766] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [11401.254121] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11401.560208] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11401.589217] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11401.647177] Lustre: lustre-MDT0000: new disk, initializing [11401.729557] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11401.739623] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11407.648319] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [11418.984289] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11419.286659] Lustre: 348799:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11419.368283] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11419.371078] Lustre: Skipped 1 previous similar message [11419.432426] Lustre: lustre-MDT0001: new disk, initializing [11419.675695] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11419.701393] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11424.637645] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [11433.110597] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11438.791028] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11445.482535] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11445.730695] Lustre: lustre-OST0000: new disk, initializing [11445.734871] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11445.745936] Lustre: 350772:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11446.956950] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11446.975035] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11447.193077] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11450.748433] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [11458.918610] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11463.340733] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11463.537967] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11467.489695] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11467.673242] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11472.869518] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11472.871105] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11472.880844] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11472.880853] Lustre: Skipped 4 previous similar messages [11472.890821] LustreError: Skipped 1 previous similar message [11472.922509] Lustre: Skipped 1 previous similar message [11483.106292] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11483.120144] Lustre: Skipped 3 previous similar messages [11484.129537] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11484.396278] Lustre: server umount lustre-OST0000 complete [11491.307903] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11491.324094] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11495.908660] Lustre: server umount lustre-MDT0000 complete [11498.470202] LustreError: 348811:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11498.507600] LustreError: 348811:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11500.427993] LustreError: 348792:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789275053 with bad export cookie 9786022089584581767 [11500.445570] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11500.457329] LustreError: 348792:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [11501.212476] Lustre: server umount lustre-MDT0001 complete [11512.136614] XFS (loop0): Unmounting Filesystem [11516.156097] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 00:51:07 (1789275067) [11523.327616] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11523.805724] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11523.810178] Lustre: Skipped 2 previous similar messages [11529.159049] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [11540.593207] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11545.892609] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [11553.492835] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11560.463825] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11566.957322] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11573.084393] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [11580.985293] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11596.260980] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11596.261829] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11596.274056] Lustre: Skipped 1 previous similar message [11596.275904] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11596.275911] Lustre: Skipped 4 previous similar messages [11596.304364] LustreError: Skipped 1 previous similar message [11599.048060] Lustre: server umount lustre-OST0000 complete [11617.248156] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11617.417022] Lustre: server umount lustre-MDT0000 complete [11622.536566] LustreError: 353309:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789275175 with bad export cookie 9786022089584582642 [11622.545064] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11622.562786] LustreError: 353309:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11623.228954] Lustre: server umount lustre-MDT0001 complete [11634.676139] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing unload_modules_local [11637.674446] Key type lgssc unregistered [11638.011862] LNet: 356961:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11638.028149] LNetError: 356961:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11638.046670] LNet: Removed LNI 192.168.203.115@tcp [11638.956155] Key type .llcrypt unregistered [11638.958261] Key type ._llcrypt unregistered [11666.098780] Key type ._llcrypt registered [11666.109674] Key type .llcrypt registered [11666.225600] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [11667.121180] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11667.169169] alg: No test for adler32 (adler32-zlib) [11668.240689] Lustre: Lustre: Build Version: 2.17.57_103_g3c4c45f [11668.577669] LNet: Added LNI 192.168.203.115@tcp [8/256/0/180] [11670.248706] Key type lgssc registered [11672.030199] Lustre: Echo OBD driver; http://www.lustre.org/ [11682.349033] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11682.376207] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11684.258799] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11690.949724] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [11694.571980] LustreError: 358526:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11694.600802] LustreError: 358526:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11699.683251] LustreError: 358527:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11701.224357] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11701.577657] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11706.540205] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [11713.849172] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11719.927972] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11727.659197] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11728.182265] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11735.483988] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [11743.386470] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11756.081595] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.15@tcp (stopping) [11757.030179] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11757.075622] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11757.086373] Lustre: Skipped 1 previous similar message [11760.923500] Lustre: server umount lustre-MDT0001 complete [11761.159555] LustreError: 359530:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.15@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11766.288359] LustreError: 358522:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.15@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11766.306123] LustreError: 358522:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11771.407404] LustreError: 359530:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.15@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11771.429946] LustreError: 359530:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11771.449221] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11771.562196] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11771.565528] Lustre: Found index 1 for lustre-MDT0001, updating log [11771.598489] Lustre: 358515:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11771.877570] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11776.046427] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [11777.014476] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11779.584117] LustreError: 362127:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11779.596700] LustreError: 362127:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11779.610922] LustreError: 358520:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11779.687836] LustreError: 362130:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.203.115@tcp: cfg command failed: rc = -17 [11779.702293] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11779.702293] [12083.750190] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.15@tcp (stopping) [12084.200761] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12084.202980] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12084.216936] Lustre: Skipped 1 previous similar message [12087.595309] Lustre: server umount lustre-MDT0001 complete [12088.842186] LustreError: 358523:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.15@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12098.091268] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12098.220327] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12098.232065] Lustre: Found index 1 for lustre-MDT0001, updating log [12098.281567] Lustre: 358516:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12098.650820] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12103.678047] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12103.681850] LustreError: 370462:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12103.716283] LustreError: 370462:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12103.730479] LustreError: 358520:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12103.745011] LustreError: 358520:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12103.871652] LustreError: 370473:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12103.883873] LustreError: 370473:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12103.897955] LustreError: 370473:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.203.115@tcp: cfg command failed: rc = -17 [12103.912077] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.203.115@tcp [12103.912077] [12104.209535] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [12410.859850] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12410.860413] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12410.871849] Lustre: Skipped 1 previous similar message [12410.876864] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12410.900534] Lustre: Skipped 3 previous similar messages [12415.972453] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12415.981294] Lustre: Skipped 2 previous similar messages [12417.083495] Lustre: server umount lustre-MDT0001 complete [12421.091848] LustreError: 358526:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12421.113745] LustreError: 358526:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [12427.869158] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12428.001158] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12428.011960] Lustre: Found index 1 for lustre-MDT0001, updating log [12428.084054] Lustre: 358515:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12428.500277] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12432.878073] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [12433.929354] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12433.963362] LustreError: 379138:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12433.976765] LustreError: 379138:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12433.991801] LustreError: 358520:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12434.005408] LustreError: 358520:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12434.093549] LustreError: 379147:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12434.119333] LustreError: 379147:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.203.115@tcp: cfg command failed: rc = -114 [12434.129519] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12434.129519] [12755.937988] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12755.944543] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12755.967710] Lustre: Skipped 1 previous similar message [12755.975622] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12755.987708] Lustre: Skipped 1 previous similar message [12760.921599] Lustre: server umount lustre-MDT0000 complete [12761.572415] LustreError: 358523:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12761.580067] LustreError: 358523:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [12764.100573] LustreError: 358508:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789276317 with bad export cookie 3813293636553498782 [12764.117039] LustreError: 358508:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12764.117669] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12766.690422] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12766.708928] Lustre: Skipped 2 previous similar messages [12770.536627] Lustre: server umount lustre-MDT0001 complete [12774.229223] Lustre: server umount lustre-OST0000 complete [12784.155940] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_hostid [12794.909549] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [12835.410102] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 01:13:06 (1789276386) [12842.496113] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12842.680414] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12842.710195] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12842.797162] Lustre: lustre-MDT0000: new disk, initializing [12842.850590] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12842.869887] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12847.739275] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [12858.945733] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12859.100196] Lustre: 389921:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12859.151690] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12859.155199] Lustre: Skipped 1 previous similar message [12859.213374] Lustre: lustre-MDT0001: new disk, initializing [12859.302052] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12859.348802] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12859.363056] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12864.106823] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [12874.130494] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12880.158954] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12887.442321] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12887.767291] Lustre: lustre-OST0000: new disk, initializing [12887.773233] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12887.782208] Lustre: 391889:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12887.910895] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12894.734199] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_default_debug -1 all [12896.861828] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12896.871759] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12897.011813] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12904.963312] Lustre: DEBUG MARKER: oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12912.613018] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12912.616474] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12912.630801] Lustre: Skipped 1 previous similar message [12912.644166] Lustre: Skipped 7 previous similar messages [12914.715340] Lustre: server umount lustre-MDT0000 complete [12917.735984] LustreError: 389929:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12917.750456] LustreError: 389929:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [12918.061925] LustreError: 390716:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789276471 with bad export cookie 3813293636553500735 [12918.065084] LustreError: MGC192.168.203.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12918.065626] LustreError: 390728:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [12918.075350] LustreError: 390716:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12918.354990] Lustre: server umount lustre-MDT0001 complete [12932.686600] Lustre: server umount lustre-OST0000 complete [12961.316227] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing set_hostid [12973.608063] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing load_modules_local [13007.818785] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12846 sec ========= 01:15:59 (1789276559) [13009.550082] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 01:16:01 (1789276561) === [13012.976515] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 01:16:04 (1789276564) === [13058.231910] Lustre: DEBUG MARKER: oleg315-server.virtnet: executing unload_modules_local [13060.896041] Key type lgssc unregistered [13061.247802] LNet: 398253:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13061.254624] LNetError: 398253:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13061.276962] LNet: Removed LNI 192.168.203.115@tcp [13062.191714] Key type .llcrypt unregistered [13062.194536] Key type ._llcrypt unregistered