[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 469113081 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2624MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001009] APIC: Switch to symmetric I/O mode setup [ 0.003216] x2apic enabled [ 0.004009] Switched APIC routing to physical x2apic. [ 0.005013] kvm-guest: setup PV IPIs [ 0.008277] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009021] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010012] pid_max: default: 32768 minimum: 301 [ 0.011140] LSM: Security Framework initializing [ 0.012052] Yama: becoming mindful. [ 0.013035] SELinux: Initializing. [ 0.014072] *** VALIDATE selinux *** [ 0.021830] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.025746] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.026149] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.027099] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028111] *** VALIDATE tmpfs *** [ 0.030171] *** VALIDATE proc *** [ 0.031242] *** VALIDATE cgroup *** [ 0.032011] *** VALIDATE cgroup2 *** [ 0.033300] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.034146] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.035009] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.036031] Spectre V2 : User space: Vulnerable [ 0.037009] Speculative Store Bypass: Vulnerable [ 0.040040] debug: unmapping init [mem 0xffffffff93a59000-0xffffffff93a60fff] [ 0.043177] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044640] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045023] ... version: 2 [ 0.046012] ... bit width: 48 [ 0.046790] ... generic registers: 4 [ 0.047013] ... value mask: 0000ffffffffffff [ 0.048016] ... max period: 00007fffffffffff [ 0.049013] ... fixed-purpose events: 3 [ 0.050019] ... event mask: 000000070000000f [ 0.051302] rcu: Hierarchical SRCU implementation. [ 0.053379] smp: Bringing up secondary CPUs ... [ 0.054473] x86: Booting SMP configuration: [ 0.055023] .... node #0, CPUs: #1 #2 #3 [ 0.058444] smp: Brought up 1 node, 4 CPUs [ 0.060013] smpboot: Max logical packages: 1 [ 0.061019] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.126017] node 0 deferred pages initialised in 63ms [ 0.129187] devtmpfs: initialized [ 0.130243] x86/mm: Memory block size: 128MB [ 0.132796] gcov: version magic: 0x41383552 [ 0.134379] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.135070] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.136229] pinctrl core: initialized pinctrl subsystem [ 0.137183] [ 0.137732] ************************************************************* [ 0.138017] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.139012] ** ** [ 0.140011] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.141014] ** ** [ 0.142011] ** This means that this kernel is built to expose internal ** [ 0.143014] ** IOMMU data structures, which may compromise security on ** [ 0.144013] ** your system. ** [ 0.145013] ** ** [ 0.146014] ** If you see this message and you are not debugging the ** [ 0.147012] ** kernel, report this immediately to your vendor! ** [ 0.148013] ** ** [ 0.149012] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.150015] ************************************************************* [ 0.151753] NET: Registered protocol family 16 [ 0.152553] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.153098] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.154063] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.155340] cpuidle: using governor menu [ 0.157731] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.160541] PCI: Using configuration type 1 for base access [ 0.162138] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.173060] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.174024] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.176073] cryptd: max_cpu_qlen set to 1000 [ 0.178229] ACPI: Added _OSI(Module Device) [ 0.179013] ACPI: Added _OSI(Processor Device) [ 0.180031] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.182035] ACPI: Added _OSI(Processor Aggregator Device) [ 0.188081] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.195355] ACPI: Interpreter enabled [ 0.197080] ACPI: PM: (supports S0 S3 S4 S5) [ 0.198012] ACPI: Using IOAPIC for interrupt routing [ 0.200095] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.204449] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.214316] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.216035] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.219021] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.223076] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.228378] acpiphp: Slot [2] registered [ 0.230159] acpiphp: Slot [5] registered [ 0.232089] acpiphp: Slot [6] registered [ 0.234136] acpiphp: Slot [7] registered [ 0.236125] acpiphp: Slot [8] registered [ 0.238133] acpiphp: Slot [9] registered [ 0.239120] acpiphp: Slot [10] registered [ 0.241135] acpiphp: Slot [3] registered [ 0.243101] acpiphp: Slot [4] registered [ 0.244097] acpiphp: Slot [11] registered [ 0.246147] acpiphp: Slot [12] registered [ 0.247072] acpiphp: Slot [13] registered [ 0.248104] acpiphp: Slot [14] registered [ 0.249077] acpiphp: Slot [15] registered [ 0.251097] acpiphp: Slot [16] registered [ 0.252093] acpiphp: Slot [17] registered [ 0.254108] acpiphp: Slot [18] registered [ 0.255096] acpiphp: Slot [19] registered [ 0.257090] acpiphp: Slot [20] registered [ 0.258267] acpiphp: Slot [21] registered [ 0.260158] acpiphp: Slot [22] registered [ 0.262153] acpiphp: Slot [23] registered [ 0.264225] acpiphp: Slot [24] registered [ 0.265126] acpiphp: Slot [25] registered [ 0.267191] acpiphp: Slot [26] registered [ 0.269151] acpiphp: Slot [27] registered [ 0.271136] acpiphp: Slot [28] registered [ 0.272186] acpiphp: Slot [29] registered [ 0.274155] acpiphp: Slot [30] registered [ 0.276222] acpiphp: Slot [31] registered [ 0.278113] PCI host bridge to bus 0000:00 [ 0.279026] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.282035] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.285037] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.288039] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.290033] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.293052] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.295260] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.298316] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.302348] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.311017] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.316861] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.319018] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.321018] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.324017] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.326573] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.328802] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.331042] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.334872] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.340014] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.352014] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.356014] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.363500] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.375025] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.384026] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.402016] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.412728] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.422071] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.426015] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.442016] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.452496] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.459023] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.465015] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.481023] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.496039] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.505016] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.510013] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.523017] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.532305] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.542024] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.552024] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.566014] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.576373] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.583015] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.590013] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.603013] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.613718] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.615366] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.618384] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.621408] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.624242] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.628766] iommu: Default domain type: Passthrough [ 0.630546] SCSI subsystem initialized [ 0.632169] ACPI: bus type USB registered [ 0.635116] usbcore: registered new interface driver usbfs [ 0.637101] usbcore: registered new interface driver hub [ 0.639083] usbcore: registered new device driver usb [ 0.641176] pps_core: LinuxPPS API ver. 1 registered [ 0.643012] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.646064] PTP clock support registered [ 0.649035] EDAC MC: Ver: 3.0.0 [ 0.651119] PCI: Using ACPI for IRQ routing [ 0.653889] NetLabel: Initializing [ 0.655014] NetLabel: domain hash size = 128 [ 0.656008] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.657072] NetLabel: unlabeled traffic allowed by default [ 0.660054] vgaarb: loaded [ 0.661143] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.663014] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.668354] clocksource: Switched to clocksource kvm-clock [ 0.768091] VFS: Disk quotas dquot_6.6.0 [ 0.770438] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.773813] *** VALIDATE ramfs *** [ 0.775017] *** VALIDATE hugetlbfs *** [ 0.776818] pnp: PnP ACPI init [ 0.779159] pnp: PnP ACPI: found 6 devices [ 0.800039] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.803565] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.806109] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.808732] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.811409] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.814031] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.817193] NET: Registered protocol family 2 [ 0.819718] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.826472] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.830764] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.836308] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.839140] TCP: Hash tables configured (established 65536 bind 65536) [ 0.842268] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.845084] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.847219] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.850253] NET: Registered protocol family 1 [ 0.853297] RPC: Registered named UNIX socket transport module. [ 0.856246] RPC: Registered udp transport module. [ 0.858196] RPC: Registered tcp transport module. [ 0.860631] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.864099] NET: Registered protocol family 44 [ 0.866053] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.868245] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.870304] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.873109] PCI: CLS 0 bytes, default 64 [ 0.875092] Unpacking initramfs... [ 2.266408] debug: unmapping init [mem 0xffff9066bcc54000-0xffff9066bffbffff] [ 2.270318] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.272440] software IO TLB: mapped [mem 0x00000000b8c54000-0x00000000bcc54000] (64MB) [ 2.274450] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.760123] Initialise system trusted keyrings [ 2.762371] Key type blacklist registered [ 2.764603] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.773671] zbud: loaded [ 2.776713] *** VALIDATE nfs *** [ 2.777986] *** VALIDATE nfs4 *** [ 2.781278] pstore: using deflate compression [ 2.787334] Platform Keyring initialized [ 2.883276] NET: Registered protocol family 38 [ 2.884669] Key type asymmetric registered [ 2.885773] Asymmetric key parser 'x509' registered [ 2.886939] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.889846] io scheduler mq-deadline registered [ 2.892384] io scheduler kyber registered [ 2.893684] io scheduler bfq registered [ 2.895235] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.897773] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.900928] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.903938] ACPI: Power Button [PWRF] [ 2.908748] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.915392] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.926436] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 2.932458] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 2.943597] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.971611] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.000915] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.006144] Non-volatile memory driver v1.3 [ 3.007809] Linux agpgart interface v0.103 [ 3.036390] virtio_blk virtio1: [vda] 149376 512-byte logical blocks (76.5 MB/72.9 MiB) [ 3.039320] vda: detected capacity change from 0 to 76480512 [ 3.053281] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.055642] vdb: detected capacity change from 0 to 1073741824 [ 3.068075] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.071908] vdc: detected capacity change from 0 to 2621440000 [ 3.083586] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.086802] vdd: detected capacity change from 0 to 2621440000 [ 3.099820] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.102812] vde: detected capacity change from 0 to 4294967296 [ 3.117433] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.122047] vdf: detected capacity change from 0 to 4294967296 [ 3.131354] libphy: Fixed MDIO Bus: probed [ 3.140365] usbcore: registered new interface driver usbserial_generic [ 3.143241] usbserial: USB Serial support registered for generic [ 3.145733] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.150432] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.151954] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.154533] mousedev: PS/2 mouse device common for all mice [ 3.157911] rtc_cmos 00:05: RTC can wake from S4 [ 3.160518] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.165391] rtc_cmos 00:05: registered as rtc0 [ 3.166774] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.167523] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.172366] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.172963] intel_pstate: CPU model not supported [ 3.178323] hid: raw HID events driver (C) Jiri Kosina [ 3.180501] usbcore: registered new interface driver usbhid [ 3.182229] usbhid: USB HID core driver [ 3.183842] drop_monitor: Initializing network drop monitor service [ 3.186402] Initializing XFRM netlink socket [ 3.188358] NET: Registered protocol family 10 [ 3.191402] Segment Routing with IPv6 [ 3.192802] NET: Registered protocol family 17 [ 3.194744] mpls_gso: MPLS GSO support [ 3.200251] RAS: Correctable Errors collector initialized. [ 3.201917] AVX version of gcm_enc/dec engaged. [ 3.203370] AES CTR mode by8 optimization enabled [ 3.272620] sched_clock: Marking stable (3272586067, 0)->(4218918941, -946332874) [ 3.276563] registered taskstats version 1 [ 3.278673] Loading compiled-in X.509 certificates [ 3.280753] zswap: loaded using pool lzo/zbud [ 3.303164] Key type big_key registered [ 3.315868] Key type encrypted registered [ 3.317550] ima: No TPM chip found, activating TPM-bypass! [ 3.319855] ima: Allocated hash algorithm: sha1 [ 3.321732] ima: No architecture policies found [ 3.323405] evm: Initialising EVM extended attributes: [ 3.325649] evm: security.selinux [ 3.326989] evm: security.ima [ 3.328386] evm: security.capability [ 3.329747] evm: HMAC attrs: 0x1 [ 3.332237] rtc_cmos 00:05: setting system clock to 2026-08-22 10:14:30 UTC (1787393670) [ 3.339163] debug: unmapping init [mem 0xffffffff94a03000-0xffffffff94bfffff] [ 3.342578] debug: unmapping init [mem 0xffffffff93782000-0xffffffff93a58fff] [ 3.353061] Write protecting the kernel read-only data: 28672k [ 3.356246] debug: unmapping init [mem 0xffffffff91e03000-0xffffffff91ffffff] [ 3.359321] debug: unmapping init [mem 0xffffffff92714000-0xffffffff927fffff] [ 3.390561] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.399514] systemd[1]: Detected virtualization kvm. [ 3.401562] systemd[1]: Detected architecture x86-64. [ 3.403865] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.427709] systemd[1]: No hostname configured. [ 3.429383] systemd[1]: Set hostname to . [ 3.431363] random: systemd: uninitialized urandom read (16 bytes read) [ 3.434099] systemd[1]: Initializing machine ID from random generator. [ 3.476504] random: ln: uninitialized urandom read (6 bytes read) [ 3.548535] random: systemd: uninitialized urandom read (16 bytes read) [ 3.550437] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 3.554199] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 3.559668] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on Journal Socket. [ OK ] Started Memstrack Anylazing Service. Starting Apply Kernel Variables... [ OK ] Reached target Initrd Root Device. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Reached target Sockets. Starting Setup Virtual Console... [ OK ] Reached target Timers. [ OK ] Reached target Slices. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.087623] device-mapper: uevent: version 1.0.3 [ 4.089397] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 4.789256] random: fast init done [ 4.794335] virtio_net virtio0 ens2: renamed from eth0 [ 4.889703] scsi host0: ata_piix [ 4.925279] scsi host1: ata_piix [ 4.929220] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 4.932240] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 8.429981] dracut-initqueue[582]: RTNETLINK answers: File exists [ 9.835207] random: crng init done [ 9.836700] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.122790] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Swap. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. Stopping udev Kernel Device Manager... [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.278064] printk: systemd: 25 output lines suppressed due to ratelimiting [ 11.527137] SELinux: Disabled at runtime. [ 11.585641] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.593856] systemd[1]: Detected virtualization kvm. [ 11.595498] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.030565] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.033894] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.038955] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.043319] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.046188] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.051658] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.057654] systemd[1]: Starting Create list of required static device nodes for the current kernel... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target rpc_pipefs.target. Mounting POSIX Message Queue File System... Mounting Huge Pages File System... Activating swap /dev/disk/by-label/SWAP... [ OK [[ 12.108506] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS 0m] Listening on Process Core Dump Socket. [ OK ] Listening on udev Control Socket. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Stopped target Switch Root. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Stopped target Initrd Root File System. Starting Remount Root and Kernel File Systems... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. Mounting Kernel Debug File System... [ OK ] Created slice system-getty.slice. [ OK ] Created slice User and Session Slice. [ OK ] Stopped target Initrd File Systems. [ OK ] Reached target Slices. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Starting Apply Kernel Variables... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [ OK ] Started Journal Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Huge Pages File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Apply Kernel Variables. Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Mounted /mnt. [ 12.422172] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Coldplug all Devices. [ OK ] Started udev Kernel Device Manager. [ 12.659666] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 12.673604] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 12.793817] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 12.805795] EDAC sbridge: Ver: 1.1.2 [ 14.415954] Key type dns_resolver registered [ 14.714570] NFS: Registering the id_resolver key type [ 14.717926] Key type id_resolver registered [ 14.719702] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Basic System. [ OK ] Reached target sshd-keygen.target. Starting Restore /run/initramfs on shutdown... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Started irqbalance daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... Starting Login Service... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg324-server login: [ 63.386793] libcfs: loading out-of-tree module taints kernel. [ 63.412964] Key type ._llcrypt registered [ 63.415146] Key type .llcrypt registered [ 63.498806] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_hostid [ 79.184487] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 80.643691] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 80.655742] alg: No test for adler32 (adler32-zlib) [ 81.974249] Lustre: Lustre: Build Version: 2.17.54_107_g42be189 [ 82.801393] LNet: Added LNI 192.168.203.124@tcp [8/256/0/180] [ 84.637296] Key type lgssc registered [ 86.350096] Lustre: Echo OBD driver; http://www.lustre.org/ [ 102.051331] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 134.956295] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 146.546094] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 146.571745] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 147.807664] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 147.843541] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 147.934721] Lustre: lustre-MDT0000: new disk, initializing [ 148.007849] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 148.032116] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 151.479978] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 162.738777] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 162.849902] Lustre: 6508:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 162.902134] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 162.908078] Lustre: Skipped 1 previous similar message [ 162.981980] Lustre: lustre-MDT0001: new disk, initializing [ 163.036228] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 163.062472] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 163.075117] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 166.895464] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 171.369411] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 180.153949] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 180.466469] Lustre: lustre-OST0000: new disk, initializing [ 180.471826] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 180.478146] Lustre: 8444:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 180.550886] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 186.318164] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 189.987550] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 190.009214] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 190.101987] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 199.942958] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 200.069557] Lustre: lustre-OST0001: new disk, initializing [ 200.072753] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 200.076980] Lustre: 9515:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 200.127259] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 206.127540] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 208.419044] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 208.433365] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 208.483260] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 217.458879] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 226.200150] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 232.325899] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing check_logdir /tmp/testlogs/ [ 237.237154] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing yml_node [ 241.892234] Lustre: DEBUG MARKER: Client: 2.17.54.107 [ 244.258764] Lustre: DEBUG MARKER: MDS: 2.17.54.107 [ 246.635659] Lustre: DEBUG MARKER: OSS: 2.17.54.107 [ 248.399201] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Sat Aug 22 06:18:33 EDT 2026 [ 264.334870] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 265.616257] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 285.156545] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 285.157643] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 285.171960] Lustre: Skipped 3 previous similar messages [ 285.183754] Lustre: Skipped 3 previous similar messages [ 287.754860] Lustre: server umount lustre-MDT0000 complete [ 290.967019] LustreError: 8443:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787393958 with bad export cookie 10202689815408607670 [ 290.971118] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 290.974913] LustreError: 8443:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 295.393398] LustreError: 6516:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 295.394676] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 295.406376] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 295.445604] LustreError: 6516:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 295.469508] Lustre: Skipped 1 previous similar message [ 297.464996] Lustre: server umount lustre-MDT0001 complete [ 301.046667] Lustre: server umount lustre-OST0000 complete [ 304.331573] Lustre: server umount lustre-OST0001 complete [ 311.245892] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_hostid [ 320.485824] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 353.393211] hrtimer: interrupt took 5102535 ns [ 354.527242] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 354.718887] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 354.790870] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 354.844170] Lustre: lustre-MDT0000: new disk, initializing [ 354.969548] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 355.001759] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 359.281761] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 371.447547] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 371.747694] Lustre: 15068:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 371.914655] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 371.925965] Lustre: Skipped 1 previous similar message [ 372.072334] Lustre: lustre-MDT0001: new disk, initializing [ 372.335438] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 372.451778] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 372.486828] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 377.982318] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 388.156848] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 394.024663] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 400.870729] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 401.270623] Lustre: lustre-OST0000: new disk, initializing [ 401.279438] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 401.286292] Lustre: 17039:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 401.416728] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 403.100464] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 403.129742] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 403.246864] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 406.836039] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 415.809939] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 420.778168] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 421.101825] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 426.098562] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 426.327251] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 429.028440] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 429.034449] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 429.035314] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 429.052123] Lustre: Skipped 1 previous similar message [ 429.064245] Lustre: Skipped 2 previous similar messages [ 434.162496] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 434.177153] Lustre: Skipped 1 previous similar message [ 434.534913] Lustre: server umount lustre-OST0000 complete [ 438.754503] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 438.764462] LustreError: Skipped 1 previous similar message [ 438.769434] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 438.781187] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 444.388429] LustreError: 15078:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 444.402522] LustreError: 15078:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 444.441771] Lustre: server umount lustre-MDT0000 complete [ 448.161348] LustreError: 17038:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787394115 with bad export cookie 10202689815408609203 [ 448.170554] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 448.175805] LustreError: 17038:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 448.576053] Lustre: server umount lustre-MDT0001 complete [ 453.915982] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 06:21:59 (1787394119) [ 461.184642] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 461.640203] LustreError: 19304:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 461.718053] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 466.635586] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 466.914438] LustreError: 19305:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 471.014912] LustreError: 19304:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 474.912830] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 479.170702] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 485.543318] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 490.304329] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 496.213433] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 502.963920] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 510.697920] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 513.953802] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 521.697227] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 521.708089] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 521.730495] Lustre: Skipped 1 previous similar message [ 521.742651] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 521.752636] Lustre: Skipped 2 previous similar messages [ 526.223832] Lustre: server umount lustre-MDT0000 complete [ 527.334087] LustreError: 21794:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 527.362616] LustreError: 21794:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 530.323512] LustreError: 19284:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787394197 with bad export cookie 10202689815408610085 [ 530.325422] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 530.357809] LustreError: 19284:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 530.755661] Lustre: server umount lustre-MDT0001 complete [ 545.847037] Lustre: server umount lustre-OST0000 complete [ 568.702404] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 579.505890] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 580.012550] LustreError: 23682:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 580.114586] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 580.119279] Lustre: Skipped 2 previous similar messages [ 584.722355] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 593.923924] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 598.699543] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 601.346749] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 607.605919] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 614.237516] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 621.880524] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 622.065648] Lustre: lustre-OST0001: new disk, initializing [ 622.070815] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 622.077334] Lustre: 25995:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 623.675486] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 623.689487] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 623.866656] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 628.586829] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 636.781695] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 639.830256] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 650.721769] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 650.733888] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 650.758411] Lustre: Skipped 2 previous similar messages [ 650.766244] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 650.769043] Lustre: Skipped 3 previous similar messages [ 656.715845] Lustre: server umount lustre-MDT0000 complete [ 656.869834] LustreError: 23679:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 656.880832] LustreError: 23679:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 660.103976] LustreError: 23663:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787394327 with bad export cookie 10202689815408610932 [ 660.106332] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 660.113596] LustreError: 23663:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 660.335910] Lustre: server umount lustre-MDT0001 complete [ 674.565185] Lustre: server umount lustre-OST0000 complete [ 688.500312] Lustre: server umount lustre-OST0001 complete [ 705.597555] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 716.715355] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 717.452911] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 717.471811] Lustre: Skipped 3 previous similar messages [ 722.390233] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 722.913034] LustreError: 29020:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 722.942784] LustreError: 29020:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 730.682750] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 735.383922] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 738.079778] Lustre: 30167:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 744.341224] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 751.082702] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 759.482635] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 766.667715] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 774.100934] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 777.486575] Lustre: 32046:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 783.337689] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 783.342282] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 783.352324] Lustre: Skipped 5 previous similar messages [ 783.357353] Lustre: Skipped 5 previous similar messages [ 789.453990] Lustre: server umount lustre-MDT0000 complete [ 792.790650] LustreError: 29001:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787394459 with bad export cookie 10202689815408612227 [ 792.796366] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 792.814105] LustreError: 29001:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 793.204678] Lustre: server umount lustre-MDT0001 complete [ 806.030666] Lustre: server umount lustre-OST0000 complete [ 808.940829] Lustre: 3647:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787394460/real 1787394460] req@ffff90660283f100 x1874218188033536/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787394476 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 809.865187] Lustre: server umount lustre-OST0001 complete [ 817.673801] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 06:28:03 (1787394483) [ 845.863300] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 855.615914] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 856.057105] LustreError: 34847:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 856.085703] LustreError: 34847:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 860.695847] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 868.797713] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 873.005969] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 875.837209] Lustre: 35995:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 882.135295] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 888.929519] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 897.665892] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 904.590539] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 911.256216] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 914.377453] Lustre: 37878:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 920.548996] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 920.562393] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 920.575303] Lustre: Skipped 2 previous similar messages [ 920.584944] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 920.591656] Lustre: Skipped 6 previous similar messages [ 925.476799] Lustre: server umount lustre-MDT0000 complete [ 928.958099] LustreError: 34827:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787394596 with bad export cookie 10202689815408613333 [ 928.965434] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 928.970424] LustreError: 34827:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 929.280254] Lustre: server umount lustre-MDT0001 complete [ 943.431343] Lustre: server umount lustre-OST0000 complete [ 957.469834] Lustre: server umount lustre-OST0001 complete [ 965.212052] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 06:30:30 (1787394630) [ 980.268942] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 992.787962] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 994.028508] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 994.042392] Lustre: Skipped 7 previous similar messages [ 1001.409515] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 1011.699847] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1016.453372] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 1019.760358] Lustre: 41170:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1027.156037] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1034.723203] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 1044.473713] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1051.037934] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 1058.484488] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1061.641177] Lustre: 43049:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1062.534223] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1062.541419] Lustre: Skipped 1 previous similar message [ 1068.514217] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1068.523860] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1068.534387] Lustre: Skipped 3 previous similar messages [ 1068.538157] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1068.546337] Lustre: Skipped 4 previous similar messages [ 1072.107692] Lustre: server umount lustre-MDT0000 complete [ 1076.119938] LustreError: 41553:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787394743 with bad export cookie 10202689815408614439 [ 1076.128076] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1076.139081] LustreError: 41553:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1082.697417] Lustre: server umount lustre-MDT0001 complete [ 1091.506012] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1101.280819] LustreError: 3643:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff90672fac6d80 x1874218188133248/t0(0) o250->MGC192.168.203.124@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1107.360664] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 1111.249570] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1116.644900] LustreError: 43924:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1116.675403] LustreError: 43924:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 35 previous similar messages [ 1117.296902] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1122.716463] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 1123.308374] Lustre: lustre-MDT0001-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 1123.311968] Lustre: Skipped 1 previous similar message [ 1130.120657] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1132.376508] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1138.023176] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1139.855380] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1148.386726] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1150.660430] Lustre: server umount lustre-MDT0000 complete [ 1154.094489] LustreError: 40003:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787394821 with bad export cookie 10202689815408615538 [ 1154.096881] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1154.102271] LustreError: 40003:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1154.455158] Lustre: server umount lustre-MDT0001 complete [ 1168.541399] Lustre: server umount lustre-OST0000 complete [ 1182.756925] Lustre: server umount lustre-OST0001 complete [ 1190.777717] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 06:34:16 (1787394856) [ 1208.013895] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 1218.695539] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1223.599908] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 1231.224681] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1236.006282] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 1238.749832] Lustre: 48807:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1246.127486] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1253.291263] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 1261.549021] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1267.919495] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 1274.544628] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1278.554397] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1293.199537] Lustre: server umount lustre-MDT0000 complete [ 1293.285865] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1293.292690] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1293.301107] Lustre: Skipped 9 previous similar messages [ 1296.588744] LustreError: 47637:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787394963 with bad export cookie 10202689815408616364 [ 1296.595992] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1296.605221] LustreError: 47637:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1296.875467] Lustre: server umount lustre-MDT0001 complete [ 1311.304418] Lustre: server umount lustre-OST0000 complete [ 1324.147869] Lustre: server umount lustre-OST0001 complete [ 1332.420294] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 06:36:37 (1787394997) [ 1334.631930] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1336.367372] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 06:36:41 (1787395001) [ 1346.716889] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_hostid [ 1355.745761] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 1373.844380] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1374.095972] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1374.179405] Lustre: lustre-MDT0000: new disk, initializing [ 1374.277277] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1377.916073] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 1385.592408] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1391.371567] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1391.677952] Lustre: lustre-OST0000: new disk, initializing [ 1391.685568] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1391.695981] Lustre: Skipped 1 previous similar message [ 1391.705401] Lustre: 54886:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1393.619568] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1393.627785] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1393.668139] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1397.448956] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 1406.206648] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1408.030150] Lustre: DEBUG MARKER: create test files [ 1439.714497] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1439.725369] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1439.735222] Lustre: Skipped 14 previous similar messages [ 1443.191437] Lustre: server umount lustre-OST0000 complete [ 1447.389427] Lustre: server umount lustre-MDT0000 complete [ 1460.195701] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing unload_modules_local [ 1463.064359] Key type lgssc unregistered [ 1463.384043] LNet: 56514:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1463.418130] LNetError: 56514:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1464.489077] LNet: Removed LNI 192.168.203.124@tcp [ 1465.300671] Key type .llcrypt unregistered [ 1465.303838] Key type ._llcrypt unregistered [ 1496.208693] Key type ._llcrypt registered [ 1496.211584] Key type .llcrypt registered [ 1496.366558] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 1497.914518] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1498.007484] alg: No test for adler32 (adler32-zlib) [ 1499.335938] Lustre: Lustre: Build Version: 2.17.54_107_g42be189 [ 1499.590752] LNet: Added LNI 192.168.203.124@tcp [8/256/0/180] [ 1501.272175] Key type lgssc registered [ 1502.847209] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1511.175743] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1511.211675] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1512.804884] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1517.817617] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 1525.332832] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1530.702374] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1531.155641] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1536.945378] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 1537.326367] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1537.454549] Lustre: 59247:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1537.463747] Lustre: 59247:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1537.479688] Lustre: 59247:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1537.493223] Lustre: 59247:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1537.511193] Lustre: 59247:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1537.517543] Lustre: 59247:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1543.046936] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1547.441660] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1553.622346] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1560.661709] Lustre: server umount lustre-OST0000 complete [ 1574.055027] Lustre: server umount lustre-MDT0000 complete [ 1583.848040] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing unload_modules_local [ 1586.202415] Key type lgssc unregistered [ 1586.460315] LNet: 60854:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1586.465625] LNetError: 60854:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1586.488980] LNet: Removed LNI 192.168.203.124@tcp [ 1587.239714] Key type .llcrypt unregistered [ 1587.246115] Key type ._llcrypt unregistered [ 1614.473360] Key type ._llcrypt registered [ 1614.475724] Key type .llcrypt registered [ 1614.644676] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 1615.991849] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1616.038865] alg: No test for adler32 (adler32-zlib) [ 1617.093265] Lustre: Lustre: Build Version: 2.17.54_107_g42be189 [ 1617.374888] LNet: Added LNI 192.168.203.124@tcp [8/256/0/180] [ 1619.032557] Key type lgssc registered [ 1620.279324] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1627.777929] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1627.796904] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1629.310825] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1633.903026] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 1640.904597] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1647.201589] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1647.683353] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1653.971772] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 1654.841947] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1654.953979] Lustre: 63689:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1654.971318] Lustre: 63689:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1654.980547] Lustre: 63689:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1654.985548] Lustre: 63689:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1654.993710] Lustre: 63689:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1655.001567] Lustre: 63689:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1661.179843] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1666.068865] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1675.234116] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1675.261142] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1679.899567] Lustre: server umount lustre-OST0000 complete [ 1683.148701] Lustre: server umount lustre-MDT0000 complete [ 1693.532314] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing unload_modules_local [ 1696.475088] Key type lgssc unregistered [ 1696.784625] LNet: 65247:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1696.793133] LNetError: 65247:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1696.816533] LNet: Removed LNI 192.168.203.124@tcp [ 1697.749678] Key type .llcrypt unregistered [ 1697.755078] Key type ._llcrypt unregistered [ 1714.654596] Key type ._llcrypt registered [ 1714.659215] Key type .llcrypt registered [ 1714.755990] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_hostid [ 1733.003483] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 1733.860951] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1733.911778] alg: No test for adler32 (adler32-zlib) [ 1735.279547] Lustre: Lustre: Build Version: 2.17.54_107_g42be189 [ 1735.698700] LNet: Added LNI 192.168.203.124@tcp [8/256/0/180] [ 1737.464410] Key type lgssc registered [ 1739.497083] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1777.274400] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1777.303349] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1778.594351] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1778.634608] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1778.736303] Lustre: lustre-MDT0000: new disk, initializing [ 1778.837173] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1778.855268] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1783.299366] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 1794.394383] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1794.522898] Lustre: 68397:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1794.595499] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1794.598877] Lustre: Skipped 1 previous similar message [ 1794.692734] Lustre: lustre-MDT0001: new disk, initializing [ 1794.834792] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1794.874149] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1794.886559] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1799.961456] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 1809.733675] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1815.044349] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1821.123901] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1821.372838] Lustre: lustre-OST0000: new disk, initializing [ 1821.377489] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1821.387548] Lustre: 70367:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1821.495514] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1828.263343] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 1828.981233] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1828.996167] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1829.139220] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1837.578722] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1842.334109] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1842.556687] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1847.155462] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1847.331870] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1849.835143] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1849.838270] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1849.853391] LustreError: Skipped 1 previous similar message [ 1849.857310] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1849.863891] Lustre: Skipped 1 previous similar message [ 1849.888187] Lustre: Skipped 1 previous similar message [ 1854.959668] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1854.965207] Lustre: Skipped 1 previous similar message [ 1855.442632] Lustre: server umount lustre-OST0000 complete [ 1860.071968] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1860.093904] Lustre: Skipped 1 previous similar message [ 1860.095514] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1865.185813] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1865.200604] Lustre: Skipped 2 previous similar messages [ 1870.305453] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1870.317618] Lustre: Skipped 1 previous similar message [ 1873.381098] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1873.740366] Lustre: server umount lustre-MDT0000 complete [ 1877.124798] LustreError: 68390:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787395544 with bad export cookie 17949694873420712012 [ 1877.134441] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1877.137910] LustreError: 68390:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1877.531557] Lustre: server umount lustre-MDT0001 complete [ 1885.208460] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 06:45:50 (1787395550) [ 1895.346981] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 1929.570365] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_hostid [ 1938.591210] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 1969.885479] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1970.092667] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1970.114070] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1970.175855] Lustre: lustre-MDT0000: new disk, initializing [ 1970.226197] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1970.241667] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1974.184498] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 1982.736633] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1982.857865] Lustre: 76385:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1982.902030] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1982.906388] Lustre: Skipped 1 previous similar message [ 1982.973992] Lustre: lustre-MDT0001: new disk, initializing [ 1983.049396] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1983.080660] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1983.093093] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1986.765838] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 1994.036493] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1998.820529] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2004.331434] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2004.610968] Lustre: lustre-OST0000: new disk, initializing [ 2004.616931] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2004.632461] Lustre: 78355:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2004.739935] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2010.159951] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 2013.249196] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2013.266277] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2013.362296] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2018.355618] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2022.883766] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2023.124890] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2027.800610] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2028.075645] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2034.147869] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2034.150370] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2034.165690] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2034.174531] Lustre: Skipped 1 previous similar message [ 2036.618634] Lustre: server umount lustre-OST0000 complete [ 2044.396903] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2044.410389] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2044.411607] Lustre: Skipped 1 previous similar message [ 2047.655315] Lustre: server umount lustre-MDT0000 complete [ 2049.509344] LustreError: 76397:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2049.520806] LustreError: 76397:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 2051.130979] LustreError: 76378:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787395718 with bad export cookie 17949694873420712894 [ 2051.154895] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2051.525305] Lustre: server umount lustre-MDT0001 complete [ 2059.625836] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 06:48:45 (1787395725) [ 2066.221761] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2066.649162] LustreError: 80766:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2066.729313] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2071.030078] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 2072.046487] LustreError: 80767:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2076.135220] LustreError: 80766:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2079.302332] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2083.618657] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 2090.573493] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2096.023690] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2103.114309] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2103.698770] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2103.706835] Lustre: Skipped 1 previous similar message [ 2109.882846] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 2117.130508] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2120.566334] Lustre: 83436:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting f5d84171-0794-498f-b3b2-d6cb7985bdd0 at adminstrative request [ 2121.190331] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2121.208929] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2131.430940] Lustre: MGS: Client f5d84171-0794-498f-b3b2-d6cb7985bdd0 (at 0@lo) reconnecting [ 2131.442605] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xf91a2062c4e44f30 to 0xf91a2062c4e451d0 [ 2131.469419] Lustre: MGC192.168.203.124@tcp: Connection restored to 0@lo (at 0@lo) [ 2158.902732] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2159.122664] Lustre: lustre-OST0001: new disk, initializing [ 2159.128659] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2159.145187] Lustre: 83830:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2165.268532] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 2165.882382] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2165.890374] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2166.026940] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2174.829459] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2181.616618] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2181.616731] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2181.625875] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2181.625884] Lustre: Skipped 3 previous similar messages [ 2181.629525] LustreError: Skipped 1 previous similar message [ 2181.680449] Lustre: Skipped 1 previous similar message [ 2184.315586] Lustre: server umount lustre-OST0001 complete [ 2186.733426] LustreError: 84687:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2186.755977] LustreError: 84687:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2191.848381] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2191.852698] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2194.290741] Lustre: server umount lustre-OST0000 complete [ 2212.320298] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2212.520222] Lustre: server umount lustre-MDT0000 complete [ 2216.141074] LustreError: 80746:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787395883 with bad export cookie 17949694873420714448 [ 2216.144525] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2216.152284] LustreError: 80746:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 2216.604909] Lustre: server umount lustre-MDT0001 complete [ 2226.202062] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 06:51:31 (1787395891) [ 2227.735449] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2229.218537] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 06:51:34 (1787395894) [ 2230.679426] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2232.197113] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 06:51:37 (1787395897) [ 2233.433068] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2234.700183] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 06:51:40 (1787395900) [ 2236.008316] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2237.482972] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 06:51:43 (1787395903) [ 2238.839107] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2243.788369] loop: module loaded [ 2243.818466] loop0: detected capacity change from 0 to 4294967296 [ 2243.882519] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2245.259182] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2248.445883] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2251.829956] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2276.404160] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2276.477487] LustreError: 87956:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2276.487537] LustreError: 87956:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2276.504138] LustreError: 87956:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2276.515665] LustreError: 87956:0:(super25.c:184:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2294.478880] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_hostid [ 2304.414274] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 2340.464393] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2340.703199] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2340.740522] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2340.854070] Lustre: lustre-MDT0000: new disk, initializing [ 2340.978429] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2340.987735] Lustre: Skipped 1 previous similar message [ 2341.025584] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2345.612314] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 2355.829744] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2355.944893] Lustre: 90820:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2356.126708] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2360.665529] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 2369.722367] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2374.981296] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2380.977429] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2381.322186] Lustre: 92787:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2387.560441] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 2388.586323] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2388.791504] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2396.529885] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2401.193548] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2401.359471] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2405.699146] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2406.049594] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2409.441757] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2409.448323] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2409.471549] Lustre: Skipped 3 previous similar messages [ 2409.477364] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2409.483136] Lustre: Skipped 10 previous similar messages [ 2422.241119] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2422.353704] Lustre: server umount lustre-OST0000 complete [ 2427.873893] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2427.883907] LustreError: Skipped 1 previous similar message [ 2431.444281] Lustre: server umount lustre-MDT0000 complete [ 2434.528224] LustreError: 90812:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787396101 with bad export cookie 17949694873420715064 [ 2434.530676] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2434.536037] LustreError: 90812:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2434.858923] Lustre: server umount lustre-MDT0001 complete [ 2442.684982] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 06:55:08 (1787396108) [ 2449.026533] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2449.502870] LustreError: 95203:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2449.518478] LustreError: 95203:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 2453.883168] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 2461.674652] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2466.191293] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 2472.279937] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2477.142331] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2483.136745] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2483.456594] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2483.464211] Lustre: Skipped 4 previous similar messages [ 2489.128230] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 2495.776793] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2499.853752] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2500.039751] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2504.329305] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2504.512570] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2510.416526] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2510.567532] Lustre: lustre-OST0001: new disk, initializing [ 2510.573751] Lustre: Skipped 2 previous similar messages [ 2510.583171] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2510.595121] Lustre: Skipped 3 previous similar messages [ 2510.601550] Lustre: 98572:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2516.101270] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 2516.567425] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2516.578549] Lustre: Skipped 2 previous similar messages [ 2516.584753] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2516.693959] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2524.211767] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2528.100326] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2528.331707] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2532.224702] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2532.415926] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2549.210798] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2571.558858] Lustre: Failing over lustre-MDT0000 [ 2571.986544] Lustre: server umount lustre-MDT0000 complete [ 2573.286804] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2573.293114] LustreError: 95931:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2573.306930] Lustre: Skipped 4 previous similar messages [ 2573.357405] LustreError: 95931:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 2589.664161] Lustre: 66620:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787396240/real 1787396240] req@00000000da3f0ab0 x1874219921312384/t0(0) o400->MGC192.168.203.124@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787396256 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2589.709569] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2599.275701] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2599.280336] LDISKFS-fs (dm-0): recovery complete [ 2599.295716] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2607.405708] LustreError: 97045:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.203.24@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2607.432714] LustreError: 97045:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 43 previous similar messages [ 2614.662308] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2617.637151] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2617.646071] Lustre: 95199:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2619.577409] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 2619.887795] Lustre: 95204:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2619.890645] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2619.906636] Lustre: 95204:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 1 previous similar message [ 2620.019958] LustreError: 101421:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2620.352090] LustreError: 101421:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2620.686673] LustreError: 101421:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2620.699638] LustreError: 101421:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 1 previous similar message [ 2621.026053] LustreError: 101421:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2621.032065] LustreError: 101421:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 1 previous similar message [ 2621.055311] Lustre: 101421:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2621.076858] Lustre: 101421:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 7 previous similar messages [ 2621.752043] LustreError: 101421:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2621.766918] LustreError: 101421:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 2 previous similar messages [ 2622.089297] LustreError: 101421:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2622.106661] LustreError: 101421:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 2 previous similar messages [ 2623.089206] Lustre: 101421:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2623.106562] Lustre: 101421:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 5 previous similar messages [ 2623.755128] LustreError: 101421:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2623.769800] LustreError: 101421:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 5 previous similar messages [ 2624.104114] LustreError: 101421:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2624.116932] LustreError: 101421:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 5 previous similar messages [ 2627.307723] Lustre: 101421:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2627.331888] Lustre: 101421:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 12 previous similar messages [ 2627.989855] LustreError: 101421:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2628.007607] LustreError: 101421:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 12 previous similar messages [ 2628.328130] LustreError: 101421:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2628.344026] LustreError: 101421:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 12 previous similar messages [ 2635.328921] Lustre: 101421:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2635.344654] Lustre: 101421:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 24 previous similar messages [ 2636.304278] LustreError: 101421:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2636.327156] LustreError: 101421:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 25 previous similar messages [ 2636.648262] LustreError: 101421:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2636.657083] LustreError: 101421:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 25 previous similar messages [ 2651.506982] Lustre: 101421:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2651.534730] Lustre: 101421:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 50 previous similar messages [ 2652.496982] LustreError: 101421:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2652.511797] LustreError: 101421:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 50 previous similar messages [ 2652.840584] LustreError: 101421:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2652.858388] LustreError: 101421:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 50 previous similar messages [ 2677.216693] Lustre: 101421:0:(ldlm_lib.c:2380:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2677.227379] Lustre: 101421:0:(ldlm_lib.c:2390:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2677.235503] Lustre: 101421:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 0d1437db-aa53-4067-8cf9-d920c31000cf@192.168.203.24@tcp [ 2677.248366] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2677.255349] Lustre: 101421:0:(ldlm_lib.c:1900:abort_req_replay_queue()) @@@ aborted: req@000000002dd36fe5 x1874219901194752/t0(8589934776) o35->0d1437db-aa53-4067-8cf9-d920c31000cf@192.168.203.24@tcp:295/0 lens 392/0 e 0 to 0 dl 1787396355 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2677.285767] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2677.302976] Lustre: lustre-MDT0000: Denying connection for new client 0d1437db-aa53-4067-8cf9-d920c31000cf (at 192.168.203.24@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2677.338332] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2677.364234] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2677.371462] Lustre: Skipped 2 previous similar messages [ 2677.374299] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2677.412881] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 2677.420356] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 2691.560703] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2691.571435] Lustre: Skipped 2 previous similar messages [ 2691.576538] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2691.581768] Lustre: Skipped 8 previous similar messages [ 2704.354331] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2704.469271] Lustre: server umount lustre-OST0000 complete [ 2706.915957] LustreError: 97022:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2706.934239] LustreError: 97022:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 11 previous similar messages [ 2708.962108] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2714.338866] Lustre: server umount lustre-OST0001 complete [ 2724.817902] Lustre: server umount lustre-MDT0000 complete [ 2728.410097] LustreError: 97046:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787396395 with bad export cookie 17949694873420766332 [ 2728.412336] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2728.419277] LustreError: 97046:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2728.714709] Lustre: server umount lustre-MDT0001 complete [ 2738.706935] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 07:00:04 (1787396404) [ 2747.676647] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2747.679941] Lustre: *** cfs_fail_loc=197, val=0*** [ 2763.169228] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2768.290091] LustreError: 103395:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.124@tcp: failed processing log, type 4: rc = -110 [ 2773.429663] Lustre: *** cfs_fail_loc=197, val=0*** [ 2773.433493] Lustre: Skipped 65 previous similar messages [ 2793.952281] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2793.960687] Lustre: Skipped 2 previous similar messages [ 2799.409280] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 2805.904393] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2807.879391] Lustre: server umount lustre-OST0000 complete [ 2814.973430] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 07:01:20 (1787396480) [ 2835.140426] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2835.782167] LustreError: 105030:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2835.824958] LustreError: 105030:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 2840.304494] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 2848.351465] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2851.988215] LustreError: 105061:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2852.001058] LustreError: 105061:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2852.008930] LustreError: 105061:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 16, retries 0, failed: rc = -5 [ 2852.266133] Lustre: server umount lustre-MDT0000 complete [ 2883.145415] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_hostid [ 2893.069761] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 2928.135822] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2928.499753] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2928.563570] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2928.689628] Lustre: lustre-MDT0000: new disk, initializing [ 2928.858923] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2934.347503] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 2944.505599] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2944.607238] Lustre: 109234:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2944.880747] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2947.982459] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 2956.813295] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2961.303689] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2967.051273] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2967.273835] Lustre: 111204:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2972.433541] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 2976.955896] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2980.194549] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2984.440769] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2984.598715] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2989.300142] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2989.547789] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2992.609321] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2992.612933] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2992.626635] LustreError: Skipped 3 previous similar messages [ 2992.637492] Lustre: Skipped 5 previous similar messages [ 2992.649446] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2992.653621] Lustre: Skipped 11 previous similar messages [ 2997.612391] Lustre: server umount lustre-OST0000 complete [ 3007.222348] Lustre: server umount lustre-MDT0000 complete [ 3010.503656] LustreError: 109226:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787396677 with bad export cookie 17949694873420776944 [ 3010.516936] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3010.520887] LustreError: 109226:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3010.956088] Lustre: server umount lustre-MDT0001 complete [ 3018.461498] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 07:04:43 (1787396683) [ 3026.477432] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing unload_modules_local [ 3028.967259] Key type lgssc unregistered [ 3029.247243] LNet: 113690:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3029.255122] LNetError: 113690:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3029.273581] LNet: Removed LNI 192.168.203.124@tcp [ 3030.062870] Key type .llcrypt unregistered [ 3030.066819] Key type ._llcrypt unregistered [ 3043.711585] Key type ._llcrypt registered [ 3043.716510] Key type .llcrypt registered [ 3043.827540] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_hostid [ 3056.071540] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 3056.868534] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3056.891326] alg: No test for adler32 (adler32-zlib) [ 3057.943341] Lustre: Lustre: Build Version: 2.17.54_107_g42be189 [ 3058.157240] LNet: Added LNI 192.168.203.124@tcp [8/256/0/180] [ 3059.840143] Key type lgssc registered [ 3060.909896] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3091.089541] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3091.107729] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3092.302401] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3092.330936] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3092.379709] Lustre: lustre-MDT0000: new disk, initializing [ 3092.433329] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3092.450916] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3095.852970] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 3103.355786] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3103.601667] Lustre: lustre-OST0000: new disk, initializing [ 3103.607513] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3103.612820] Lustre: Skipped 1 previous similar message [ 3103.617525] Lustre: 117839:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3103.685732] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3105.525995] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3105.533559] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3105.555298] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3107.725771] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 3115.967179] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3116.154864] Lustre: lustre-OST0001: new disk, initializing [ 3116.160835] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3116.170355] Lustre: 118704:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3116.243207] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3121.545213] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 3125.344479] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3125.353309] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3125.390668] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3130.340176] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3130.349598] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3130.353801] Lustre: Skipped 1 previous similar message [ 3130.361657] Lustre: Skipped 1 previous similar message [ 3135.222924] Lustre: server umount lustre-MDT0000 complete [ 3140.278325] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3143.902487] LustreError: 118703:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787396811 with bad export cookie 13370018529782094203 [ 3143.906083] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3143.913220] LustreError: 118703:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3143.984423] Lustre: server umount lustre-OST0000 complete [ 3146.986566] Lustre: server umount lustre-OST0001 complete [ 3153.208468] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 07:06:59 (1787396819) [ 3170.609991] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 07:07:16 (1787396836) [ 3178.772711] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 3196.179191] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 07:07:41 (1787396861) [ 3230.321476] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_hostid [ 3239.278735] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 3270.208242] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3270.399782] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3270.432577] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3270.509969] Lustre: lustre-MDT0000: new disk, initializing [ 3270.557290] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3270.572874] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3274.912508] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 3285.199557] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3285.373330] Lustre: 126448:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3285.440089] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3285.443225] Lustre: Skipped 1 previous similar message [ 3285.504461] Lustre: lustre-MDT0001: new disk, initializing [ 3285.581870] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3285.609673] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3285.619900] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3289.551071] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 3297.571398] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3302.298145] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3307.125538] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3307.348242] Lustre: lustre-OST0000: new disk, initializing [ 3307.351111] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3307.356252] Lustre: 128416:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3307.409503] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3311.866753] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 3315.246049] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3315.260070] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3315.379661] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3320.147728] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3324.031581] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3324.194259] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3327.741804] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3327.905887] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3331.045865] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3331.047203] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3331.070919] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3331.083047] Lustre: Skipped 1 previous similar message [ 3335.581373] Lustre: server umount lustre-OST0000 complete [ 3341.285225] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3341.286192] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3341.302990] Lustre: Skipped 2 previous similar messages [ 3344.902884] Lustre: server umount lustre-MDT0000 complete [ 3347.967342] LustreError: 126442:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787397015 with bad export cookie 13370018529782095428 [ 3347.985918] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3347.986642] LustreError: 126442:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3348.356172] Lustre: server umount lustre-MDT0001 complete [ 3356.724404] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 07:10:22 (1787397022) [ 3363.692081] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3364.070054] LustreError: 130828:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3364.085232] LustreError: 130828:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3364.130983] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3367.873301] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 3369.445535] LustreError: 130829:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3374.565569] LustreError: 130828:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3375.261398] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3379.041233] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 3384.681417] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3389.501156] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3395.189687] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3401.032624] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 3406.645444] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3415.845655] Lustre: *** cfs_fail_loc=159, val=0*** [ 3415.859938] Lustre: Skipped 1 previous similar message [ 3431.257416] Lustre: lustre-MDT0000: Client 1c90c795-39df-454a-9e63-4214313a5e75 (at 192.168.203.24@tcp) reconnecting [ 3432.533681] Lustre: *** cfs_fail_loc=159, val=0*** [ 3432.535607] Lustre: Skipped 4 previous similar messages [ 3448.179749] Lustre: lustre-MDT0000: Client 1c90c795-39df-454a-9e63-4214313a5e75 (at 192.168.203.24@tcp) reconnecting [ 3454.243432] Lustre: server umount lustre-OST0000 complete [ 3460.068330] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3460.073713] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3460.078147] Lustre: Skipped 1 previous similar message [ 3460.092372] Lustre: Skipped 2 previous similar messages [ 3465.191580] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3465.200361] Lustre: Skipped 1 previous similar message [ 3471.328963] Lustre: 115049:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787397122/real 1787397122] req@000000003a89d1b2 x1874221308413696/t0(0) o400->lustre-OST0000-osc-MDT0001@0@lo:28/4 lens 224/224 e 0 to 1 dl 1787397138 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3471.343659] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3472.352386] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3472.576168] Lustre: server umount lustre-MDT0000 complete [ 3473.376120] Lustre: 115047:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787397124/real 1787397124] req@00000000b7485955 x1874221308413952/t0(0) o13->lustre-OST0000-osc-MDT0001@0@lo:7/4 lens 224/368 e 0 to 1 dl 1787397140 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3476.093725] LustreError: 130808:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787397143 with bad export cookie 13370018529782096310 [ 3476.096205] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3476.115819] LustreError: 130808:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3476.459481] Lustre: server umount lustre-MDT0001 complete [ 3489.482376] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing unload_modules_local [ 3492.538725] Key type lgssc unregistered [ 3492.832125] LNet: 134604:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3492.847558] LNetError: 134604:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3492.871904] LNet: Removed LNI 192.168.203.124@tcp [ 3493.916759] Key type .llcrypt unregistered [ 3493.923885] Key type ._llcrypt unregistered [ 3519.255548] Key type ._llcrypt registered [ 3519.258664] Key type .llcrypt registered [ 3519.363610] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 3520.450869] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3520.512323] alg: No test for adler32 (adler32-zlib) [ 3521.668515] Lustre: Lustre: Build Version: 2.17.54_107_g42be189 [ 3521.904440] LNet: Added LNI 192.168.203.124@tcp [8/256/0/180] [ 3523.592152] Key type lgssc registered [ 3524.894940] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3532.725971] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3532.782050] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3534.339329] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3539.021661] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 3544.545591] LustreError: 136021:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3544.564441] LustreError: 136021:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3548.358696] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3548.641797] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3553.862573] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 3561.768714] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3567.549719] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3573.918308] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3574.355436] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3579.532866] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3581.647800] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 3590.826918] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3604.727556] Lustre: *** cfs_fail_loc=159, val=0*** [ 3620.164433] Lustre: lustre-MDT0000: Client d97db9d0-9424-44af-8d0b-1fee65693d81 (at 192.168.203.24@tcp) reconnecting [ 3622.370250] Lustre: *** cfs_fail_loc=159, val=0*** [ 3638.687980] Lustre: lustre-MDT0001: Client d97db9d0-9424-44af-8d0b-1fee65693d81 (at 192.168.203.24@tcp) reconnecting [ 3640.331510] Lustre: *** cfs_fail_loc=159, val=0*** [ 3640.335609] Lustre: Skipped 3 previous similar messages [ 3656.526659] Lustre: lustre-MDT0001: Client d97db9d0-9424-44af-8d0b-1fee65693d81 (at 192.168.203.24@tcp) reconnecting [ 3660.860957] Lustre: 139057:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3663.506028] Lustre: *** cfs_fail_loc=159, val=0*** [ 3663.511281] Lustre: Skipped 5 previous similar messages [ 3680.087674] Lustre: lustre-MDT0000: Client f752c607-7f4d-47c0-8c43-2770c8ed4b6f (at 192.168.203.24@tcp) reconnecting [ 3681.694328] Lustre: *** cfs_fail_loc=159, val=0*** [ 3681.696235] Lustre: Skipped 13 previous similar messages [ 3697.516317] Lustre: lustre-MDT0000: Client f752c607-7f4d-47c0-8c43-2770c8ed4b6f (at 192.168.203.24@tcp) reconnecting [ 3699.052401] Lustre: 139346:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3718.124428] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3718.133740] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3718.154519] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3722.212932] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3722.214318] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3722.217155] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3722.226136] Lustre: Skipped 1 previous similar message [ 3723.903150] Lustre: server umount lustre-OST0000 complete [ 3732.450550] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3732.467311] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3732.480335] Lustre: Skipped 1 previous similar message [ 3732.502934] Lustre: Skipped 1 previous similar message [ 3734.412572] Lustre: server umount lustre-MDT0000 complete [ 3737.572921] LustreError: 136995:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3737.602608] LustreError: 136995:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3738.027671] LustreError: 136001:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787397405 with bad export cookie 14407310539339174843 [ 3738.036814] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3738.060938] LustreError: 136001:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3738.513816] Lustre: server umount lustre-MDT0001 complete [ 3750.257766] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing unload_modules_local [ 3753.043851] Key type lgssc unregistered [ 3753.368642] LNet: 140429:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3753.376766] LNetError: 140429:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3753.394244] LNet: Removed LNI 192.168.203.124@tcp [ 3754.218257] Key type .llcrypt unregistered [ 3754.220639] Key type ._llcrypt unregistered [ 3778.233070] Key type ._llcrypt registered [ 3778.237093] Key type .llcrypt registered [ 3778.373972] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 3779.882690] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3779.934248] alg: No test for adler32 (adler32-zlib) [ 3781.369453] Lustre: Lustre: Build Version: 2.17.54_107_g42be189 [ 3781.947574] LNet: Added LNI 192.168.203.124@tcp [8/256/0/180] [ 3783.833208] Key type lgssc registered [ 3785.378573] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3793.385727] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3793.404424] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3794.782429] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3798.285899] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 3804.139095] LustreError: 141847:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3804.157584] LustreError: 141847:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3805.878299] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3806.319895] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3810.245781] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 3817.525737] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3823.544321] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3830.391775] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3830.841231] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3831.869871] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 3837.530880] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 3839.035267] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 3844.818175] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3851.327748] Lustre: 144425:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3854.265233] Lustre: 144473:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3855.822682] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 3856.614606] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 3860.096790] Lustre: 144620:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 3860.969329] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 3861.778671] Lustre: 144718:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3867.618518] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3867.629259] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3867.659977] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3869.670227] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3869.672093] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3869.680257] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3869.711246] Lustre: Skipped 1 previous similar message [ 3872.129615] Lustre: server umount lustre-OST0000 complete [ 3877.860516] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3877.870132] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3877.894872] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3879.906332] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3879.908972] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3885.028329] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3885.039824] Lustre: Skipped 2 previous similar messages [ 3890.144464] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3890.474385] Lustre: server umount lustre-MDT0000 complete [ 3894.226309] LustreError: 141827:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787397561 with bad export cookie 12550227137644592338 [ 3894.245639] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3894.248334] LustreError: 141827:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3894.669311] Lustre: server umount lustre-MDT0001 complete [ 3906.533977] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing unload_modules_local [ 3910.179869] Key type lgssc unregistered [ 3910.570523] LNet: 145777:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3910.586286] LNetError: 145777:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3910.622112] LNet: Removed LNI 192.168.203.124@tcp [ 3911.645129] Key type .llcrypt unregistered [ 3911.646779] Key type ._llcrypt unregistered [ 3939.356159] Key type ._llcrypt registered [ 3939.359459] Key type .llcrypt registered [ 3939.451912] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 3940.917405] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3940.976013] alg: No test for adler32 (adler32-zlib) [ 3942.000949] Lustre: Lustre: Build Version: 2.17.54_107_g42be189 [ 3942.359132] LNet: Added LNI 192.168.203.124@tcp [8/256/0/180] [ 3944.080310] Key type lgssc registered [ 3945.243098] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3954.154972] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3954.181880] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3955.696484] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3960.175127] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 3964.900271] LustreError: 147194:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3964.915523] LustreError: 147194:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3970.018771] LustreError: 147193:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3970.024660] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3970.406687] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3974.048599] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 3981.104523] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3986.798235] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3992.814714] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3993.102703] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3995.192409] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 3998.935546] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 4000.298717] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4005.176944] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4011.404289] Lustre: *** cfs_fail_loc=159, val=0*** [ 4027.731374] Lustre: lustre-MDT0000: Client db51b4ed-9047-458c-a277-32956f1e2e4d (at 192.168.203.24@tcp) reconnecting [ 4046.323369] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4046.327636] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4046.330634] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4046.338418] Lustre: Skipped 1 previous similar message [ 4046.343793] LustreError: Skipped 1 previous similar message [ 4050.495218] Lustre: server umount lustre-OST0000 complete [ 4056.546343] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4056.550120] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4056.560886] Lustre: Skipped 1 previous similar message [ 4056.578129] Lustre: Skipped 2 previous similar messages [ 4060.510514] Lustre: server umount lustre-MDT0000 complete [ 4061.678431] LustreError: 147194:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4061.697992] LustreError: 147194:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4063.669021] LustreError: 147174:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787397730 with bad export cookie 16129016583874601461 [ 4063.672092] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4063.681299] LustreError: 147174:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4063.955438] Lustre: server umount lustre-MDT0001 complete [ 4072.329480] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing unload_modules_local [ 4074.538167] Key type lgssc unregistered [ 4074.793713] LNet: 150859:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4074.800486] LNetError: 150859:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4074.823495] LNet: Removed LNI 192.168.203.124@tcp [ 4075.590602] Key type .llcrypt unregistered [ 4075.593719] Key type ._llcrypt unregistered [ 4096.286342] Key type ._llcrypt registered [ 4096.294615] Key type .llcrypt registered [ 4096.373925] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 4097.252042] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4097.315351] alg: No test for adler32 (adler32-zlib) [ 4098.382648] Lustre: Lustre: Build Version: 2.17.54_107_g42be189 [ 4098.626564] LNet: Added LNI 192.168.203.124@tcp [8/256/0/180] [ 4100.280240] Key type lgssc registered [ 4101.511572] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4109.318124] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4109.329377] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4110.709596] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4114.550050] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 4120.044765] LustreError: 152255:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4120.066543] LustreError: 152255:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4121.473738] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4121.773573] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4125.552340] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 4132.715230] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4138.261185] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4143.751218] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4144.049890] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4146.097873] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4149.332945] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 4155.324440] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4155.701330] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4173.589291] Lustre: lustre-OST0000: evicting be135919-84bd-452a-b5b1-b358efa0279e (at 192.168.203.24@tcp) by administrative request [ 4173.596426] Lustre: Skipped 1 previous similar message [ 4201.446411] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4201.449474] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4201.472465] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4201.499617] Lustre: Skipped 1 previous similar message [ 4206.562780] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4206.575946] Lustre: Skipped 2 previous similar messages [ 4211.682414] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4211.688210] Lustre: Skipped 1 previous similar message [ 4215.264768] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4215.462298] Lustre: server umount lustre-OST0000 complete [ 4218.851065] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4218.861557] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4218.876339] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4221.921924] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4224.981307] Lustre: server umount lustre-MDT0000 complete [ 4227.043690] LustreError: 154949:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4227.061218] LustreError: 154949:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4228.209974] LustreError: 152237:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787397895 with bad export cookie 13047661601839236141 [ 4228.213893] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4228.220246] LustreError: 152237:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4228.563815] Lustre: server umount lustre-MDT0001 complete [ 4237.545307] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing unload_modules_local [ 4239.789126] Key type lgssc unregistered [ 4240.026659] LNet: 156324:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4240.035411] LNetError: 156324:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4240.052321] LNet: Removed LNI 192.168.203.124@tcp [ 4240.734273] Key type .llcrypt unregistered [ 4240.736741] Key type ._llcrypt unregistered [ 4306.311771] Key type ._llcrypt registered [ 4306.315038] Key type .llcrypt registered [ 4306.434618] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 4307.590565] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4307.617124] alg: No test for adler32 (adler32-zlib) [ 4308.685469] Lustre: Lustre: Build Version: 2.17.54_107_g42be189 [ 4308.957077] LNet: Added LNI 192.168.203.124@tcp [8/256/0/180] [ 4310.658110] Key type lgssc registered [ 4311.821553] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4319.367866] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4319.398219] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4320.912489] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4325.088743] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 4331.490861] LustreError: 158825:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4331.510853] LustreError: 158825:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4333.349291] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4333.753856] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4338.182522] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 4345.214455] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4350.027849] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4355.816981] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4356.181619] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4358.259740] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4362.504418] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 4366.480274] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4370.029706] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4379.105128] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4379.118572] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4379.133954] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4382.182387] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4382.192194] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4382.194268] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4383.439651] Lustre: server umount lustre-OST0000 complete [ 4389.867497] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4389.877179] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4389.906777] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4389.922846] Lustre: Skipped 1 previous similar message [ 4392.421358] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4392.421979] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4393.878846] Lustre: server umount lustre-MDT0000 complete [ 4397.543677] LustreError: 158820:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4397.566632] LustreError: 158820:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4397.939995] LustreError: 160675:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787398065 with bad export cookie 2526994567660061550 [ 4397.947056] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4397.962024] LustreError: 160675:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4398.300642] Lustre: server umount lustre-MDT0001 complete [ 4409.182906] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing unload_modules_local [ 4411.620673] Key type lgssc unregistered [ 4411.907281] LNet: 162365:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4411.918381] LNetError: 162365:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4411.953618] LNet: Removed LNI 192.168.203.124@tcp [ 4412.666047] Key type .llcrypt unregistered [ 4412.670061] Key type ._llcrypt unregistered [ 4434.404360] Key type ._llcrypt registered [ 4434.408283] Key type .llcrypt registered [ 4434.513657] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 4435.461181] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4435.587742] alg: No test for adler32 (adler32-zlib) [ 4436.623260] Lustre: Lustre: Build Version: 2.17.54_107_g42be189 [ 4436.926803] LNet: Added LNI 192.168.203.124@tcp [8/256/0/180] [ 4438.600179] Key type lgssc registered [ 4439.800560] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4453.049864] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 07:28:38 (1787398118) [ 4469.525723] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_hostid [ 4480.392280] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 4513.672194] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4513.697066] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4514.951271] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4515.020566] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4515.179946] Lustre: lustre-MDT0000: new disk, initializing [ 4515.330040] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4515.360873] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4519.527763] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 4529.998406] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4530.187747] Lustre: 166626:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4530.288270] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4530.292384] Lustre: Skipped 1 previous similar message [ 4530.376227] Lustre: lustre-MDT0001: new disk, initializing [ 4530.504143] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4530.542938] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4530.562144] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4535.681415] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 4544.456651] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4549.238555] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4555.506402] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4555.814556] Lustre: lustre-OST0000: new disk, initializing [ 4555.818322] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4555.823557] Lustre: 168595:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4555.936300] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4562.039512] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 4562.470834] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4562.483707] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4562.646478] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4571.541500] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4583.393314] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4583.401729] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4583.435090] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4587.264608] Lustre: server umount lustre-OST0000 complete [ 4592.097467] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4592.106469] LustreError: Skipped 1 previous similar message [ 4592.115151] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4592.139055] Lustre: Skipped 1 previous similar message [ 4592.148634] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4592.151289] Lustre: Skipped 1 previous similar message [ 4593.635213] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4593.639467] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4597.314224] Lustre: server umount lustre-MDT0000 complete [ 4600.749763] LustreError: 166618:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787398267 with bad export cookie 948599730399630964 [ 4600.761162] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4600.764340] LustreError: 166618:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4601.204403] Lustre: server umount lustre-MDT0001 complete [ 4612.035661] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing unload_modules_local [ 4615.018072] Key type lgssc unregistered [ 4615.385178] LNet: 170500:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4615.403545] LNetError: 170500:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4615.438702] LNet: Removed LNI 192.168.203.124@tcp [ 4616.291657] Key type .llcrypt unregistered [ 4616.306203] Key type ._llcrypt unregistered [ 4638.194670] Key type ._llcrypt registered [ 4638.197326] Key type .llcrypt registered [ 4638.300567] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 4639.375791] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4639.406609] alg: No test for adler32 (adler32-zlib) [ 4640.457499] Lustre: Lustre: Build Version: 2.17.54_107_g42be189 [ 4640.657227] LNet: Added LNI 192.168.203.124@tcp [8/256/0/180] [ 4642.296707] Key type lgssc registered [ 4643.313725] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4650.701574] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4650.724936] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4652.096866] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4657.146066] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 4662.242548] LustreError: 171917:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4662.267331] LustreError: 171917:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4665.732132] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4666.092191] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4671.226808] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 4678.550867] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4683.460531] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4688.703034] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4688.956981] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4694.775179] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 4697.155598] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4702.078183] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4708.415977] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4708.443471] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4708.459421] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4708.465798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4708.474371] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4709.447976] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4709.464208] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4709.473845] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4709.488032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4709.500466] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4710.320165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4711.120914] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4711.125341] Lustre: Skipped 1 previous similar message [ 4711.156313] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4711.175176] Lustre: Skipped 1 previous similar message [ 4711.182990] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4711.188064] Lustre: Skipped 1 previous similar message [ 4711.191474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4711.208926] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4711.217435] Lustre: Skipped 1 previous similar message [ 4712.073414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4712.914543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4713.738426] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4713.744736] Lustre: Skipped 2 previous similar messages [ 4713.763876] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4713.789901] Lustre: Skipped 2 previous similar messages [ 4713.795823] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4713.811235] Lustre: Skipped 2 previous similar messages [ 4713.815033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4713.828966] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4713.838371] Lustre: Skipped 2 previous similar messages [ 4714.672074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4715.462660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4716.376650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4717.189352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4718.086592] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4718.090737] Lustre: Skipped 4 previous similar messages [ 4718.101309] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4718.109426] Lustre: Skipped 4 previous similar messages [ 4718.117347] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4718.124308] Lustre: Skipped 4 previous similar messages [ 4718.129941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4718.137482] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4718.141434] Lustre: Skipped 4 previous similar messages [ 4719.127070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4719.969605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4720.918711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4721.764898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4722.632906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4723.514777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4724.298575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4725.166701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4726.087504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4727.114982] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4727.122502] Lustre: Skipped 9 previous similar messages [ 4727.141614] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4727.160722] Lustre: Skipped 9 previous similar messages [ 4727.166880] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4727.170764] Lustre: Skipped 9 previous similar messages [ 4727.174877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4727.186987] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4727.195381] Lustre: Skipped 9 previous similar messages [ 4728.197870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4729.092893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4730.184993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4731.135164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4732.159546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4733.067562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4733.994770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4734.918680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4735.932618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4736.865829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4737.839396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4738.845474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4739.717916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4740.682119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4741.508576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4742.355545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4743.051977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4743.934416] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4743.946430] Lustre: Skipped 17 previous similar messages [ 4743.969729] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4743.983571] Lustre: Skipped 17 previous similar messages [ 4743.993867] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4744.003193] Lustre: Skipped 17 previous similar messages [ 4744.007138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4744.021354] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4744.039923] Lustre: Skipped 17 previous similar messages [ 4745.027653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4746.161765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4746.939210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4747.829504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4748.623051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4749.670857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4750.655697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4751.493507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4752.246931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4753.231307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4754.354704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4755.218579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4756.146297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4756.928032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4757.791730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4758.550752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4759.342904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4760.230659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4761.064247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4762.099212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4763.024787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4763.905628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4764.737514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4765.652920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4766.606852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4767.475946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4768.433930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4769.411255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4770.306211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4771.182525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4772.209805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4773.102500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4774.120143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4775.054004] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4776.036639] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4776.042018] Lustre: Skipped 34 previous similar messages [ 4776.060943] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4776.069858] Lustre: Skipped 34 previous similar messages [ 4776.074341] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4776.080529] Lustre: Skipped 34 previous similar messages [ 4776.087713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4776.099972] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4776.103207] Lustre: Skipped 34 previous similar messages [ 4776.948978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4777.922569] LustreError: 171936:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4777.932987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4777.940604] LustreError: 171936:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4778.860848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4779.793724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4780.692444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4781.840674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4782.782305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4783.739828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4784.708670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4785.647853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4786.603103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4787.510386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4788.370625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4789.240960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4790.097378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4791.033220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4791.873282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4792.791660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4793.661610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4794.497880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4795.300048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4796.305174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4797.144531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4798.077655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4798.931814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4799.861636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4800.844309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4801.699064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4802.637561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4803.694410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4804.589667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4805.593315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4806.654357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4807.504845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4808.211843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4809.200207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4810.122435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4811.167557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4812.027802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4812.930600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4813.909587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4814.905524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4815.892437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4816.847579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4817.593574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4818.429762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4819.390452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4820.357550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4821.234413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4822.128516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4822.839874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4823.691994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4824.598721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4825.427984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4826.326685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4827.259966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4828.021926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4828.945249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4829.673950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4830.452983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4831.308155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4832.165606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4833.064741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4833.892969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4834.652086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4835.471200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4836.400597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4837.244261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4838.127599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4839.062156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4839.988642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4840.723834] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4840.728098] Lustre: Skipped 71 previous similar messages [ 4840.739789] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4840.756348] Lustre: Skipped 71 previous similar messages [ 4840.762882] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4840.765967] Lustre: Skipped 71 previous similar messages [ 4840.778244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4840.807884] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4840.822731] Lustre: Skipped 71 previous similar messages [ 4841.646520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4842.716601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4843.596666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4844.677830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4845.446321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4846.254458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4847.062519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4847.946414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4848.777815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4849.865094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4850.883223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4851.677152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4852.541481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4853.556128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4854.399107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4855.389269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4856.504796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4857.352095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4858.079820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4859.052770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4860.069183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4861.161648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4861.944316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4863.051485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4863.941548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4865.144842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4866.629606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4867.761038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4868.754436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4870.166138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4871.340349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4872.411147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4873.235399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4874.160323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4875.072846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4876.040947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4876.914043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4878.103497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4879.108885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4880.243477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4881.631632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4882.855293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4883.771757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4885.002449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4885.786799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4887.000092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4887.915588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4888.719918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4889.589456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4890.731681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4891.618557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4892.522628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4893.450922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4894.448478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4895.424440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4896.366480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4897.314411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4898.221155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4898.975274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4899.817564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4900.633191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4901.476843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4902.240176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4903.052595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4903.720055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4904.573799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4905.341627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4906.113429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4906.900356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4907.745342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4908.484360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4909.155726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4910.028237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4910.839895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4911.595911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4912.448163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4913.216987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4914.025456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4914.791623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4915.524771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4916.312534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4917.367915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4918.447219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4919.523785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4920.638605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4921.907125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4923.014871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4924.144473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4925.014942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4925.969696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4926.912220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4927.859910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4928.713064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4929.495848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4930.328399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4931.154806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4932.157523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4933.136235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4934.096056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4934.997712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4935.966412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4936.745749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4937.483735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4938.320873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4939.176931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4939.793905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4940.596397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4941.395363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4942.193061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4942.948764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4943.782812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4944.555900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4945.232721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4945.882402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4946.719872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4947.647969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4948.588823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4949.548801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4950.454092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4951.170789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4952.015987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4952.854040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4953.669127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4954.541904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4955.378068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4956.231357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4957.081775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4957.911059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4958.678421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4959.564552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4960.499196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4961.369590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4962.367484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4963.239034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4964.092580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4965.150501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4965.941566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4966.794122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4967.716515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4968.539797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4969.549900] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4969.562861] Lustre: Skipped 140 previous similar messages [ 4969.579070] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4969.590038] Lustre: Skipped 140 previous similar messages [ 4969.596559] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4969.600023] Lustre: Skipped 140 previous similar messages [ 4969.604172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4969.613111] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4969.618216] Lustre: Skipped 140 previous similar messages [ 4970.366966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4971.362900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4972.268817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4973.205160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4974.217199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4975.107649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4975.988770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4977.178241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4978.373844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4979.377464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4980.212267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4981.040820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4981.861721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4982.551051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4983.287372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4984.125373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4985.177610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4986.049226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4987.161096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4988.179890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4989.128702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4989.981625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4990.947411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4992.057762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4993.177971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4994.289800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4995.396480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4996.463239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.551133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4998.424746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4999.173681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5000.009072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5000.913159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.963900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5002.881302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5003.741644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5004.962760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5006.100916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.159587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5008.220988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.064900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5010.025340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.055112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5012.108172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5012.939991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.915246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5014.795223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5015.635967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5016.568143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.456189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5018.525915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5019.496592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.566028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5021.351950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.241652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5023.163785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.284275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5025.168680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5026.101309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5026.959307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.825571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5028.738267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.770635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5030.616945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5031.679765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5032.618049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.527234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.407760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.321987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.363056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5037.261723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.190535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5039.141648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.013942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.884725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.800115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.717873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.754595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.585994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.502951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.229745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.183207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.121415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.056648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.857430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.696738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.525096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.421852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.267170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.101277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.942198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.733394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.587646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.378584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.065475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.843458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.641268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.445129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.390910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.278476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.098134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.943541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.740937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.659207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.497358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.286908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.938876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.767218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.616598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.524352] LustreError: 171936:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5070.531515] LustreError: 171936:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5070.545859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.513310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.301314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.143515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.063208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.085579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.990692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.813818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.902904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.093838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.123903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.179542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.093982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.032362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.981976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.900580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.851074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.818143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.740218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.702269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.589184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.433936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.286708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.250613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.026754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.981975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.979615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.838694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.802173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.597958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.553895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.590146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.564624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.676239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.764344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.706668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.772597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.734533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.705613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.674137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.476506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.438452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.248482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.277721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.054422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.932078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.565064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.506699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.388410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.319458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.510700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.351480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.261427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.227229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.333918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.259302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.485569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.494473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.408546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.581964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.715564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.591889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.594180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.542930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.606498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.703333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.450929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.309955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.153028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.217863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.047727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.977331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.912588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.985879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.842698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.809185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.855251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.916065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.905295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.839243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.716187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.569428] LustreError: 171936:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5147.580778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.588083] LustreError: 171936:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5148.560788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.734481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.506216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.390563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.111591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.944533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.758769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.619046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.249914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.993217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.689480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.516610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.354926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.252788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.137323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.012332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.725996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.692565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.531442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.359807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.193375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.068785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.932917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.177276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.057537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.149357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.990769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.705585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.459808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.230040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.011699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.977469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.918917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.762929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.821211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.710688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.468943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.589871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.553884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.747809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.642968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.513361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.344429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.103236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.896292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.693096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.801760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.558879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.444088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.379829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.625019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.587574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.576060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.406232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.368232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.300704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.277419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.145601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.126790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.948869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.854043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.651609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.589108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.501753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.489480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.346768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.344393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.317334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.337066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.186585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.227669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.274081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.469196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.395082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.234196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.118643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.935837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.723739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.588551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.494540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.240674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.219852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.105797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.971756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.665701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.606536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.468952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.297927] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5226.303726] Lustre: Skipped 278 previous similar messages [ 5226.313549] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5226.319864] Lustre: Skipped 278 previous similar messages [ 5226.322977] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5226.335303] Lustre: Skipped 278 previous similar messages [ 5226.339883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.347378] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5226.355707] Lustre: Skipped 278 previous similar messages [ 5227.044531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.761814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.497917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.343842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.035590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.736850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.438533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.245367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.958621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.807989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.691627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.467080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.245165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.174264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.053514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.918921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.772182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.784571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.654506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.527777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.342890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.239089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.149491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.098749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.825922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.756652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.413608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.340699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.282345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.194626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.114680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.155785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.078000] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.022501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.720087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.530726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.484140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.418372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.256705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.237457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.040769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.997315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.971486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.899439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.901615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.757348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.728375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.682340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.642556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.451659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.282226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.205966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.073865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.116686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.186167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.174261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.111697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.895625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.963848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.940124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.883378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.773829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.663283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.365601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.099976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.865368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.573017] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.239313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.056286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.854254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.683078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.370645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.269653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.205546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.051605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.900283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.727491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.558829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.465051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.304495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.098251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.895400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.679029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.440387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.289070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.083982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.814129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.485825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.293504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.034156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.947855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.682450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.478636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.204615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.973662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.796642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.738695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.596910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.522993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.414556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.245139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.122673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.210082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.950513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.733607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.573759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.534799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.399746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.370086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.343566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.220475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.042846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.774711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.657419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.404592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.283087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.209876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.146789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.943056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.798352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.734348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.645726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.473803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.507625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.456627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.358960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.409184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.189211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.100719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.926227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.812317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.666702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.712688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.642429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.570211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.379864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.349435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.049917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.890088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.757404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.607968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.494434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.427526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.344095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.321859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.158197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.153466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.093460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.056325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.046387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.990744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.990117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.974146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.129652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.045235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.240660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.314527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.367561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.462729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.614433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.802447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.744813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.750825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.745537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.818187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.813536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.724958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.676170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.691625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.688133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.645505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.609484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.542907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.425936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.340399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.108208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.836993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.697115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.625024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.513247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.498744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.393078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.231628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.272674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.293526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.060968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.912706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.773059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.622366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.430260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.380628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.132690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.057756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.825481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.739887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.617388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.519562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.421683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.259760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.037763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.820399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.581425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.520887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.495650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.350876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.159494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.975060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.815795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.774709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.562571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.524774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.373352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.232955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.047786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.885131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.731880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.560394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.532676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.544078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.480781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.309969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.168806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.276943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.478332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.560107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.497102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.866948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.972609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.812183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.204413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.272526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.479954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.660884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.838734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.814549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.794452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.799934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.786699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.950448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.809559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.852103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.091267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.225095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.495071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.500739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.459642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.719930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.177216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.468080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.359219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.360488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.622558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.066910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.605352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.914696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.121248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.145901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.131243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.075614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.137078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.271748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.951091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.784795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.677365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.407713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.224585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.025880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.870183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.855058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.871443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.811855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.835255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.545359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.323890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.138709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.237154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.156955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.052514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.857828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.742335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.821875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.703041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.658663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.933683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.831150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.822290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.593191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.453904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.462546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.255642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.067826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.095241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.167523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.085931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.165636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.043150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.012368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.858297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.766867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.615244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.678536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.693382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.585678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.426702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.382821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.342817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.481380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.520716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.553927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.682133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.733554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.574735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.669778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.760388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.757243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.753613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.752786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.706200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.746884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.625994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.472703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.445515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.366517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.517045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.270479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.053372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.949918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.081965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.041560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.875564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.758438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.582608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.623125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.543447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.605202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.475261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.529191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.487915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.436436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.312103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.399388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.384055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.302182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.275683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.146477] LustreError: 171936:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5545.157195] LustreError: 171936:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5545.161553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.981064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.776488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.553182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.544225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.525221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.528199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.410754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.492956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.424477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.515965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.472395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.447773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.366635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.240567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.125378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.166168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.188297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.235871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.237505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.122453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.249173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.164789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.369697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.347081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.305975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.199783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.259965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.017808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.892778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.666627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.352250] LustreError: 171936:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5574.371188] LustreError: 171936:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5574.378183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.183201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.963156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.886242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.621749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.253848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.242322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.171677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.046513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.087545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.046031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.163514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.171780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.144988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.082064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.031604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.051124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.991718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.114283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.897542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.924405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.611801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.227944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.050462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.049995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.807250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.893350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.631546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.579455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.473609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.617388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.706266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.405669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.093935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.766887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.331885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.077606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.914477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.822184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.687841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.426651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.337831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.336989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.976821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.909625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.764626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.734156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.732340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.688563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.569331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.389164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.244951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.118939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.064267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.169954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.050281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.171592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.271517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.753076] LustreError: 173739:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 5960.764596] LustreError: 171936:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6548.454918] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6548.461376] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6548.474222] Lustre: Skipped 434 previous similar messages [ 6548.506523] Lustre: Skipped 1 previous similar message [ 6552.623949] Lustre: server umount lustre-OST0000 complete [ 6558.690908] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6558.702102] Lustre: Skipped 1 previous similar message [ 6562.964598] Lustre: server umount lustre-MDT0000 complete [ 6563.814432] LustreError: 171916:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6563.851054] LustreError: 171916:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6566.889783] LustreError: 171896:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787400234 with bad export cookie 7371750542168397242 [ 6566.897943] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6566.899257] LustreError: 171896:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6567.262923] Lustre: server umount lustre-MDT0001 complete [ 6577.753850] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing unload_modules_local [ 6580.607783] Key type lgssc unregistered [ 6581.039722] LNet: 227163:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6581.056323] LNetError: 227163:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6581.097603] LNet: Removed LNI 192.168.203.124@tcp [ 6582.137409] Key type .llcrypt unregistered [ 6582.142507] Key type ._llcrypt unregistered [ 6607.235831] Key type ._llcrypt registered [ 6607.238611] Key type .llcrypt registered [ 6607.352962] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 6608.529380] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6608.596900] alg: No test for adler32 (adler32-zlib) [ 6609.581720] Lustre: Lustre: Build Version: 2.17.54_107_g42be189 [ 6609.906682] LNet: Added LNI 192.168.203.124@tcp [8/256/0/180] [ 6611.592152] Key type lgssc registered [ 6612.978544] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6620.199866] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6620.216971] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6621.646138] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6625.928675] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 6630.886278] LustreError: 228586:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6630.902425] LustreError: 228586:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6633.955182] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6634.434781] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6639.325591] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 6646.579881] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6652.187878] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6659.624352] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6660.098975] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6666.361339] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 6666.764278] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 6674.450361] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6682.729852] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6684.838574] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6684.848468] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6684.856990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6684.868740] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6684.869081] LustreError: 228605:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 6686.693116] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6686.699533] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6686.714516] Lustre: Skipped 1 previous similar message [ 6691.667796] Lustre: lustre-OST0000: Not available for connect from 192.168.203.24@tcp (stopping) [ 6691.681735] Lustre: Skipped 1 previous similar message [ 6692.983600] Lustre: server umount lustre-OST0000 complete [ 6704.904978] LustreError: 228605:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 6726.666642] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6727.037487] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6733.636771] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 6741.371237] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6742.378113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6742.412120] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6742.418100] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6743.162975] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 6748.532858] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff945c98a31800.ost_server_uuid 50 [ 6749.943085] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff945c98a31800.ost_server_uuid in FULL state after 0 sec [ 6756.612365] Lustre: server umount lustre-OST0000 complete [ 6762.474565] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6762.486309] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6762.502791] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6762.516880] Lustre: Skipped 2 previous similar messages [ 6768.099693] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6768.113774] Lustre: Skipped 2 previous similar messages [ 6771.168539] LustreError: 228605:0:(client.c:1381:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@0000000046821346 x1874225032925184/t0(0) o7->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 264/240 e 0 to 0 dl 0 ref 2 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 6771.432309] Lustre: server umount lustre-MDT0000 complete [ 6773.221873] LustreError: 229589:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6773.246679] LustreError: 229589:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6774.054371] Lustre: 227897:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787400425/real 1787400425] req@00000000126f2175 x1874225032922240/t0(0) o400->lustre-OST0000-osc-MDT0001@0@lo:28/4 lens 224/224 e 0 to 1 dl 1787400441 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 6774.096181] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6774.116020] Lustre: Skipped 1 previous similar message [ 6775.195106] LustreError: 228567:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787400442 with bad export cookie 1832965722682201435 [ 6775.208378] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6775.627769] Lustre: server umount lustre-MDT0001 complete [ 6789.810873] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing unload_modules_local [ 6793.441517] Key type lgssc unregistered [ 6793.842726] LNet: 233796:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6793.850295] LNetError: 233796:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6793.889246] LNet: Removed LNI 192.168.203.124@tcp [ 6794.698698] Key type .llcrypt unregistered [ 6794.702413] Key type ._llcrypt unregistered [ 6825.521273] Key type ._llcrypt registered [ 6825.524536] Key type .llcrypt registered [ 6825.648774] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 6827.026851] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6827.065512] alg: No test for adler32 (adler32-zlib) [ 6828.222559] Lustre: Lustre: Build Version: 2.17.54_107_g42be189 [ 6828.598283] LNet: Added LNI 192.168.203.124@tcp [8/256/0/180] [ 6830.280156] Key type lgssc registered [ 6831.793037] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6840.910036] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6840.926073] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6842.383844] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6845.894617] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 6852.595685] LustreError: 235523:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6852.610790] LustreError: 235523:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6853.380219] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6853.671931] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6857.057699] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 6859.581572] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 6865.363580] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6865.730267] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6867.777524] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 6870.994816] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 6879.104440] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6879.271825] Lustre: lustre-OST0001: new disk, initializing [ 6879.275672] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 6879.281173] Lustre: 237831:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6879.365098] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6881.907579] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 6881.916951] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 6882.026198] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 6885.027233] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 6896.151533] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6902.063991] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 6918.123132] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6918.131415] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6918.156231] Lustre: Skipped 2 previous similar messages [ 6918.178307] Lustre: Skipped 3 previous similar messages [ 6922.484845] Lustre: server umount lustre-MDT0000 complete [ 6923.235877] LustreError: 238482:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6923.257659] LustreError: 238482:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 6926.328032] LustreError: 235504:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787400593 with bad export cookie 1172093172752478866 [ 6926.329888] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6926.341957] LustreError: 235504:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6926.661036] Lustre: server umount lustre-MDT0001 complete [ 6939.108097] Lustre: server umount lustre-OST0000 complete [ 6952.675735] Lustre: server umount lustre-OST0001 complete [ 6972.448736] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 6980.411728] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6980.831268] LustreError: 240937:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6980.911794] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6984.581147] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 6986.214862] LustreError: 240937:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6991.332730] LustreError: 240938:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6992.440974] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6992.863281] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6997.090494] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 6999.641765] Lustre: 242080:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7005.593048] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7007.021904] LustreError: 242434:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7007.043770] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 7011.701038] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 7017.482497] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:129) [ 7019.670286] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7019.989786] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7020.002521] Lustre: Skipped 1 previous similar message [ 7025.146430] Lustre: mylustre-OST0001: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:33) [ 7026.348859] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 7033.810883] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7037.137684] Lustre: 243952:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7049.187771] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7049.194269] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7049.208630] Lustre: Skipped 1 previous similar message [ 7049.218717] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7050.730307] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7050.738916] Lustre: mylustre-MDT0000-lwp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7050.746534] Lustre: Skipped 1 previous similar message [ 7050.757504] Lustre: Skipped 2 previous similar messages [ 7054.317283] Lustre: server umount mylustre-MDT0000 complete [ 7055.849417] LustreError: 242828:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7055.871711] LustreError: 242828:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 7057.816115] LustreError: 240912:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787400724 with bad export cookie 1172093172752481722 [ 7057.827861] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7057.830953] LustreError: 240912:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7058.209234] Lustre: server umount mylustre-MDT0001 complete [ 7072.300075] Lustre: server umount mylustre-OST0000 complete [ 7085.109492] Lustre: server umount mylustre-OST0001 complete [ 7104.283355] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 7114.771771] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7115.328356] LustreError: 246260:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7115.426853] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7120.315402] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 7128.456225] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7133.020916] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 7135.797035] Lustre: 247402:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7141.855269] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7147.992210] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 7153.610529] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:161) [ 7155.985154] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7160.338825] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 7160.354226] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:65) [ 7162.313465] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 7169.706437] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7173.651184] Lustre: 249277:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7185.378275] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7185.398546] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7185.424658] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7187.513451] Lustre: server umount tfs-MDT0000 complete [ 7191.027036] LustreError: 249043:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7191.052534] LustreError: 249043:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 7191.294838] LustreError: 246237:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787400858 with bad export cookie 1172093172752483899 [ 7191.299257] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7191.305056] LustreError: 246237:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7191.685414] Lustre: server umount tfs-MDT0001 complete [ 7205.432752] Lustre: server umount tfs-OST0000 complete [ 7218.169687] Lustre: server umount tfs-OST0001 complete [ 7237.216335] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 7246.973605] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7247.596614] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7247.606628] Lustre: Skipped 3 previous similar messages [ 7251.770481] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 7259.754996] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7263.668467] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 7266.069248] Lustre: 252670:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7271.480949] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7272.814263] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 7277.111576] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 7284.472718] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7285.769431] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:97) [ 7285.779214] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:193) [ 7290.419549] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 7297.495845] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7300.947224] Lustre: 254540:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7308.050249] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 08:16:13 (1787400973) [ 7326.698693] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7326.703360] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7326.704114] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7326.712877] Lustre: Skipped 4 previous similar messages [ 7326.716120] Lustre: Skipped 3 previous similar messages [ 7330.428483] Lustre: server umount lustre-MDT0000 complete [ 7331.814933] LustreError: 251526:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7331.835779] LustreError: 251526:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 9 previous similar messages [ 7333.777399] LustreError: 253849:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787401000 with bad export cookie 1172093172752485817 [ 7333.786452] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7333.788953] LustreError: 253849:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7336.931315] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7336.962082] Lustre: Skipped 2 previous similar messages [ 7336.973223] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7336.976019] Lustre: Skipped 3 previous similar messages [ 7348.193563] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7348.536862] Lustre: server umount lustre-MDT0001 complete [ 7358.464627] Lustre: server umount lustre-OST0000 complete [ 7376.352534] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7376.720171] Lustre: server umount lustre-OST0001 complete [ 7384.813737] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_hostid [ 7397.243349] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 7431.471808] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 7431.471808] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7431.471808] [ 7431.502100] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7431.821980] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7431.850881] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7431.924725] Lustre: lustre-MDT0000: new disk, initializing [ 7432.100865] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7432.106731] Lustre: Skipped 3 previous similar messages [ 7432.121143] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7436.828636] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 7447.400444] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 7447.400444] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7447.400444] [ 7447.468536] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7447.626510] Lustre: 257724:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7447.701246] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7447.705073] Lustre: Skipped 1 previous similar message [ 7447.791926] Lustre: lustre-MDT0001: new disk, initializing [ 7448.000046] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7448.013163] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7452.653418] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 7463.586898] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7463.924848] Lustre: lustre-OST0000: new disk, initializing [ 7463.929516] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7463.939285] Lustre: 259310:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7465.795793] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7465.822171] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7465.943078] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7471.107536] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 7482.524349] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7482.863392] Lustre: lustre-OST0001: new disk, initializing [ 7482.867350] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7482.874054] Lustre: 260182:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7484.361590] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7484.386129] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7484.519623] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7490.164117] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 7500.960580] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 08:19:26 (1787401166) [ 7516.309247] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 08:19:41 (1787401181) [ 7532.514251] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7532.532086] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7532.534323] LustreError: Skipped 1 previous similar message [ 7532.559600] Lustre: Skipped 1 previous similar message [ 7532.571888] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7532.590849] Lustre: Skipped 7 previous similar messages [ 7537.994902] Lustre: server umount lustre-OST0000 complete [ 7545.322292] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7548.415594] Lustre: server umount lustre-MDT0000 complete [ 7552.216020] LustreError: 257718:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787401219 with bad export cookie 1172093172752487595 [ 7552.218286] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7552.225794] LustreError: 257718:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7558.624842] Lustre: server umount lustre-MDT0001 complete [ 7569.866904] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing unload_modules_local [ 7580.389296] Lustre: server umount lustre-OST0001 complete [ 7587.260312] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_hostid [ 7601.135116] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 7638.642891] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7638.906867] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7638.937159] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7639.038294] Lustre: lustre-MDT0000: new disk, initializing [ 7639.120886] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7643.705839] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 7654.594281] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7654.708494] Lustre: 264740:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7654.957552] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7660.320978] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 7670.723991] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7676.033794] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7683.046129] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7683.325450] Lustre: lustre-OST0000: new disk, initializing [ 7683.328766] Lustre: Skipped 1 previous similar message [ 7683.332430] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7683.337952] Lustre: Skipped 2 previous similar messages [ 7683.343414] Lustre: 266706:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7685.069058] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7685.076927] Lustre: Skipped 1 previous similar message [ 7685.084966] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7685.212191] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7689.879608] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 7699.228436] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7705.575105] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7705.577750] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7705.588111] LustreError: Skipped 1 previous similar message [ 7705.589375] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7705.589380] Lustre: Skipped 7 previous similar messages [ 7705.618307] Lustre: Skipped 5 previous similar messages [ 7711.310812] Lustre: server umount lustre-OST0000 complete [ 7729.124436] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7729.351377] Lustre: server umount lustre-MDT0000 complete [ 7731.173410] LustreError: 265624:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7731.191678] LustreError: 265624:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 18 previous similar messages [ 7732.729220] LustreError: 266705:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787401399 with bad export cookie 1172093172752489065 [ 7732.735246] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7732.742571] LustreError: 266705:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7733.261796] Lustre: server umount lustre-MDT0001 complete [ 7744.662300] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing unload_modules_local [ 7747.997384] Key type lgssc unregistered [ 7748.289512] LNet: 268466:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7748.295727] LNetError: 268466:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7749.368223] LNet: Removed LNI 192.168.203.124@tcp [ 7750.611354] Key type .llcrypt unregistered [ 7750.614091] Key type ._llcrypt unregistered [ 7783.607377] Key type ._llcrypt registered [ 7783.609612] Key type .llcrypt registered [ 7783.735889] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 7784.715864] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7784.842483] alg: No test for adler32 (adler32-zlib) [ 7786.094943] Lustre: Lustre: Build Version: 2.17.54_107_g42be189 [ 7786.307726] LNet: Added LNI 192.168.203.124@tcp [8/256/0/180] [ 7787.920395] Key type lgssc registered [ 7789.695497] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7798.681106] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7798.713579] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7800.669632] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7806.719464] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 7810.020564] LustreError: 269933:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7810.033489] LustreError: 269933:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7815.144512] LustreError: 269934:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7816.350192] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7816.628052] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7821.030577] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 7827.292942] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7831.582912] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7837.614131] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7837.907557] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7843.508910] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 7845.104152] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 7849.562699] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7850.349933] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 7855.586298] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7855.597415] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7855.611528] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7855.616429] Lustre: Skipped 1 previous similar message [ 7855.628982] Lustre: Skipped 1 previous similar message [ 7860.706584] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7860.719526] Lustre: Skipped 1 previous similar message [ 7865.839622] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7865.852014] Lustre: Skipped 1 previous similar message [ 7866.337093] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7866.479306] Lustre: server umount lustre-OST0000 complete [ 7870.945658] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7870.962658] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7870.968389] Lustre: Skipped 1 previous similar message [ 7876.071989] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7876.080590] Lustre: Skipped 1 previous similar message [ 7876.680698] Lustre: server umount lustre-MDT0000 complete [ 7881.153516] LustreError: 269914:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787401548 with bad export cookie 16858292016392578501 [ 7881.156796] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7881.181239] LustreError: 269914:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7881.753593] Lustre: server umount lustre-MDT0001 complete [ 7894.793039] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing unload_modules_local [ 7897.284632] Key type lgssc unregistered [ 7897.498109] LNet: 273521:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7897.510188] LNetError: 273521:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7897.539818] LNet: Removed LNI 192.168.203.124@tcp [ 7898.334414] Key type .llcrypt unregistered [ 7898.336311] Key type ._llcrypt unregistered [ 7915.559518] Key type ._llcrypt registered [ 7915.567952] Key type .llcrypt registered [ 7915.723328] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 7916.805171] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7916.873956] alg: No test for adler32 (adler32-zlib) [ 7917.999530] Lustre: Lustre: Build Version: 2.17.54_107_g42be189 [ 7918.271297] LNet: Added LNI 192.168.203.124@tcp [8/256/0/180] [ 7920.040312] Key type lgssc registered [ 7921.466698] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7930.977283] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7931.012733] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7946.592487] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 7951.712434] LustreError: 274715:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.124@tcp: failed processing log, type 4: rc = -110 [ 7978.403826] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7984.582097] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 7991.714720] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7993.843411] Lustre: server umount lustre-OST0000 complete [ 8008.350195] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing unload_modules_local [ 8011.664327] Key type lgssc unregistered [ 8011.989847] LNet: 276345:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8012.002341] LNetError: 276345:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8013.039315] LNet: Removed LNI 192.168.203.124@tcp [ 8014.399746] Key type .llcrypt unregistered [ 8014.403442] Key type ._llcrypt unregistered [ 8054.908715] Key type ._llcrypt registered [ 8054.919154] Key type .llcrypt registered [ 8055.067916] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 8056.654515] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8056.896314] alg: No test for adler32 (adler32-zlib) [ 8057.910021] Lustre: Lustre: Build Version: 2.17.54_107_g42be189 [ 8058.251954] LNet: Added LNI 192.168.203.124@tcp [8/256/0/180] [ 8059.993482] Key type lgssc registered [ 8061.363364] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8075.335554] loop0: detected capacity change from 0 to 204800000 [ 8075.540737] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8075.548656] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8075.813485] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8076.669676] loop0: detected capacity change from 0 to 204800000 [ 8076.785610] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8076.795431] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8077.250428] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8078.071539] loop0: detected capacity change from 0 to 204800000 [ 8078.225709] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8078.233381] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8078.303402] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8079.240882] loop0: detected capacity change from 0 to 204800000 [ 8079.322093] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8080.391215] loop0: detected capacity change from 0 to 204800000 [ 8080.428649] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8083.750843] loop0: detected capacity change from 0 to 204800000 [ 8083.769528] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8087.951360] loop0: detected capacity change from 0 to 204800000 [ 8087.984461] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8091.986516] loop0: detected capacity change from 0 to 204800000 [ 8092.036533] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8096.086775] loop0: detected capacity change from 0 to 204800000 [ 8096.167292] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8096.193319] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8096.272725] Lustre: 279760:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8096.287056] Lustre: 279760:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8096.305591] Lustre: 279760:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8096.320462] Lustre: 279760:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8096.333142] Lustre: 279760:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8096.343765] Lustre: 279760:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8096.358643] Lustre: 279760:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8096.369777] Lustre: 279760:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8096.380382] Lustre: 279760:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8096.390386] Lustre: 279760:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8096.400412] Lustre: 279760:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8096.411652] Lustre: 279760:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8096.421973] Lustre: 279760:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8096.435587] Lustre: 279760:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8101.697615] Lustre: server umount MGS complete [ 8102.589043] loop0: detected capacity change from 0 to 204800000 [ 8102.668711] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8103.058904] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8103.092806] Lustre: lustre-MDT0000: Aborting client recovery [ 8103.096527] LustreError: 280134:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0000-osd: get update log duration 0, retries 0, failed: rc = -108 [ 8103.841374] loop1: detected capacity change from 0 to 204800000 [ 8103.942565] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8104.041662] Lustre: 280191:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8104.062552] Lustre: 280191:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8104.086393] Lustre: 280191:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8104.105137] Lustre: 280191:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8104.118917] Lustre: 280191:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8104.133416] Lustre: 280191:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8104.143397] Lustre: 280191:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8104.162946] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8104.339947] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8104.386913] Lustre: lustre-MDT0001: Aborting client recovery [ 8105.191843] loop2: detected capacity change from 0 to 204800000 [ 8105.308710] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8105.388047] Lustre: 280270:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8105.411377] Lustre: 280270:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8105.423729] Lustre: 280270:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8105.445913] Lustre: 280270:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8105.467383] Lustre: 280270:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8105.475526] Lustre: 280270:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8105.482578] Lustre: 280270:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8105.577405] Lustre: Found index 0 for lustre-OST0000, updating log [ 8105.604598] Lustre: 280288:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8105.687805] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8105.725923] Lustre: lustre-OST0000: Aborting recovery [ 8106.684420] loop3: detected capacity change from 0 to 204800000 [ 8106.897903] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8106.983393] Lustre: 280352:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8107.000168] Lustre: 280352:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8107.007375] Lustre: 280352:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8107.014419] Lustre: 280352:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8107.021358] Lustre: 280352:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8107.033483] Lustre: 280352:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8107.050560] Lustre: 280352:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8107.093492] Lustre: Found index 1 for lustre-OST0001, updating log [ 8107.157336] Lustre: 280358:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8107.259221] Lustre: lustre-OST0001: Aborting recovery [ 8112.639819] LustreError: 280135:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 9, retries 0, failed: rc = -108 [ 8113.647714] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8113.651602] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8113.740703] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8113.756105] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8114.658585] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8114.663339] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8115.088204] LustreError: 280102:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1734us: rc = -2 [ 8115.602657] LustreError: 280103:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 454us: rc = -2 [ 8115.617915] LustreError: 280103:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 22 previous similar messages [ 8130.022052] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8130.051904] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8130.058165] Lustre: Skipped 3 previous similar messages [ 8134.448810] Lustre: server umount lustre-MDT0000 complete [ 8135.138075] LustreError: 280105:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8135.154688] LustreError: 280105:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 8135.360621] LustreError: 280086:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787401802 with bad export cookie 6227875799579796716 [ 8135.373295] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8135.376284] LustreError: 280086:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8140.257132] LustreError: 280701:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8140.257526] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8140.281772] LustreError: 280701:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8140.319174] Lustre: Skipped 3 previous similar messages [ 8140.327375] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8140.334305] Lustre: Skipped 1 previous similar message [ 8141.875322] Lustre: server umount lustre-MDT0001 complete [ 8157.152567] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8157.253877] Lustre: server umount lustre-OST0000 complete [ 8164.526055] Lustre: server umount lustre-OST0001 complete [ 8173.250344] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 08:30:38 (1787401838) [ 8202.364479] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_hostid [ 8212.042135] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 8247.932941] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8248.201333] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8248.235930] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8248.323867] Lustre: lustre-MDT0000: new disk, initializing [ 8248.449680] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8248.454408] Lustre: Skipped 1 previous similar message [ 8248.469183] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8253.235520] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 8263.735958] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8263.904596] Lustre: 284431:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8263.977597] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8263.981766] Lustre: Skipped 1 previous similar message [ 8264.103758] Lustre: lustre-MDT0001: new disk, initializing [ 8264.284292] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8264.352684] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8264.374200] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8269.967184] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 8279.462588] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8284.961338] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8291.131691] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8291.519080] Lustre: lustre-OST0000: new disk, initializing [ 8291.522691] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8291.530610] Lustre: 286399:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8291.705889] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8297.668026] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8297.685560] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8297.829205] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8299.175994] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 8308.914189] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8314.631704] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8325.959896] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8337.239236] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8347.361355] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8390.113884] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8390.124206] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8390.133424] Lustre: Skipped 1 previous similar message [ 8390.137692] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8390.143654] Lustre: Skipped 1 previous similar message [ 8394.566491] Lustre: server umount lustre-OST0000 complete [ 8400.353539] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8400.362881] Lustre: Skipped 1 previous similar message [ 8400.371665] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8400.378474] Lustre: Skipped 1 previous similar message [ 8404.305253] Lustre: server umount lustre-MDT0000 complete [ 8405.481059] LustreError: 284441:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8405.502197] LustreError: 284441:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8408.296616] LustreError: 284421:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787402075 with bad export cookie 6227875799579799439 [ 8408.298476] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8408.311835] LustreError: 284421:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8408.723754] Lustre: server umount lustre-MDT0001 complete [ 8417.413655] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8421.622792] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 8425.724345] Lustre: server umount MGS complete [ 8435.928450] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8436.476822] LustreError: 290417:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8436.505683] LustreError: 290430:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8436.565149] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8440.953227] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 8441.831207] LustreError: 290418:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8448.479651] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8453.034418] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 8460.147904] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8465.154448] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8471.263497] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8471.828332] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8471.840823] Lustre: Skipped 1 previous similar message [ 8478.544142] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 8485.773767] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8517.101625] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8517.105320] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8517.120044] Lustre: Skipped 2 previous similar messages [ 8517.138035] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8517.147947] LustreError: Skipped 1 previous similar message [ 8521.802056] Lustre: server umount lustre-OST0000 complete [ 8525.793203] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8525.800875] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8525.819044] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8525.834915] Lustre: Skipped 1 previous similar message [ 8531.849302] Lustre: server umount lustre-MDT0000 complete [ 8532.469234] LustreError: 290418:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8532.498478] LustreError: 290418:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8535.548346] LustreError: 290397:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787402202 with bad export cookie 6227875799579801392 [ 8535.556676] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8535.561054] LustreError: 290397:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8535.834631] Lustre: server umount lustre-MDT0001 complete [ 8546.002487] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing unload_modules_local [ 8548.770824] Key type lgssc unregistered [ 8549.108530] LNet: 294359:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8549.135035] LNetError: 294359:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8549.153159] LNet: Removed LNI 192.168.203.124@tcp [ 8549.996725] Key type .llcrypt unregistered [ 8550.001026] Key type ._llcrypt unregistered [ 8583.288279] Key type ._llcrypt registered [ 8583.290040] Key type .llcrypt registered [ 8583.405836] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_hostid [ 8596.833272] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 8597.666700] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8597.786455] alg: No test for adler32 (adler32-zlib) [ 8598.966717] Lustre: Lustre: Build Version: 2.17.54_107_g42be189 [ 8599.230835] LNet: Added LNI 192.168.203.124@tcp [8/256/0/180] [ 8600.944202] Key type lgssc registered [ 8602.102137] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8633.632513] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8633.677461] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8634.814282] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8634.830018] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8634.888512] Lustre: lustre-MDT0000: new disk, initializing [ 8634.932253] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8634.946379] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8638.698665] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 8648.459983] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8648.609263] Lustre: 298316:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8648.689288] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8648.695499] Lustre: Skipped 1 previous similar message [ 8648.787340] Lustre: lustre-MDT0001: new disk, initializing [ 8648.907140] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8648.970632] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8648.983285] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8653.262092] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 8661.719178] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8667.196666] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8673.625926] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8673.943445] Lustre: lustre-OST0000: new disk, initializing [ 8673.947460] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8673.952799] Lustre: 300283:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8674.036069] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8679.934277] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 8683.073925] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8683.096037] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8683.199607] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8689.086007] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8694.719960] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8704.983227] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8710.254749] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8755.174164] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8755.175563] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8755.186055] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8755.188481] LustreError: Skipped 1 previous similar message [ 8755.217082] Lustre: Skipped 1 previous similar message [ 8757.917400] Lustre: server umount lustre-OST0000 complete [ 8761.825537] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8761.842118] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8761.858215] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8761.863098] Lustre: Skipped 1 previous similar message [ 8765.411307] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8765.412386] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8765.439206] Lustre: Skipped 1 previous similar message [ 8767.459238] Lustre: server umount lustre-MDT0000 complete [ 8770.492058] LustreError: 301943:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787402437 with bad export cookie 5609364262587583586 [ 8770.495515] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8770.503109] LustreError: 301943:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8770.766271] Lustre: server umount lustre-MDT0001 complete [ 8777.672596] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8781.520572] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 8785.407027] Lustre: server umount MGS complete [ 8794.611093] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8795.037578] LustreError: 304162:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8795.088670] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8798.556587] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 8800.230365] LustreError: 304149:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8800.242565] LustreError: 304149:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8805.357238] LustreError: 304150:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8807.053727] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8807.421333] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8811.776832] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 8819.192272] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8824.240610] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8830.616038] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8830.948692] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8836.567814] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 8843.321944] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8872.929860] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8872.954233] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8872.977474] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8876.011044] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8877.640231] Lustre: server umount lustre-OST0000 complete [ 8884.201033] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8884.223178] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8884.249847] Lustre: Skipped 1 previous similar message [ 8884.253553] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8884.264711] Lustre: Skipped 2 previous similar messages [ 8887.944746] Lustre: server umount lustre-MDT0000 complete [ 8890.337609] LustreError: 304149:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8890.352553] LustreError: 304149:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8891.504659] LustreError: 304131:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787402558 with bad export cookie 5609364262587585434 [ 8891.510602] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8891.514157] LustreError: 304131:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8891.934374] Lustre: server umount lustre-MDT0001 complete [ 8902.012463] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing unload_modules_local [ 8904.623689] Key type lgssc unregistered [ 8904.910053] LNet: 308093:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8904.921604] LNetError: 308093:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8904.952622] LNet: Removed LNI 192.168.203.124@tcp [ 8905.817205] Key type .llcrypt unregistered [ 8905.819760] Key type ._llcrypt unregistered [ 8925.512400] Key type ._llcrypt registered [ 8925.519730] Key type .llcrypt registered [ 8925.681506] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_hostid [ 8938.016286] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 8938.773426] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8938.915113] alg: No test for adler32 (adler32-zlib) [ 8939.944828] Lustre: Lustre: Build Version: 2.17.54_107_g42be189 [ 8940.197107] LNet: Added LNI 192.168.203.124@tcp [8/256/0/180] [ 8941.872632] Key type lgssc registered [ 8943.081182] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8965.619442] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 8978.991336] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8979.017465] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8980.325181] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8980.345965] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8980.437035] Lustre: lustre-MDT0000: new disk, initializing [ 8980.575695] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8980.597604] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8985.638622] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 8995.954651] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8996.267809] Lustre: lustre-OST0000: new disk, initializing [ 8996.272158] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8996.277690] Lustre: Skipped 1 previous similar message [ 8996.287102] Lustre: 312134:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8996.392775] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9001.483726] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9001.489600] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9001.518102] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9001.962746] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 9815.533544] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9815.553331] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9819.943372] Lustre: server umount lustre-MDT0000 complete [ 9823.430168] LustreError: 311320:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787403490 with bad export cookie 3559134749452656907 [ 9823.437573] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9823.442359] LustreError: 311320:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9823.594648] Lustre: server umount lustre-OST0000 complete [ 9846.977145] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_hostid [ 9855.606900] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [ 9888.128225] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9888.351840] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9888.374604] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9888.485854] Lustre: lustre-MDT0000: new disk, initializing [ 9888.575701] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9888.590958] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9892.490863] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 9902.259864] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9902.397738] Lustre: 316461:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9902.459694] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9902.463883] Lustre: Skipped 1 previous similar message [ 9902.570854] Lustre: lustre-MDT0001: new disk, initializing [ 9902.638588] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9902.669497] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9902.694589] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9907.571672] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 9916.772084] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9922.682504] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9930.003210] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9930.362943] Lustre: lustre-OST0000: new disk, initializing [ 9930.368083] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9930.373883] Lustre: 318428:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9930.467339] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9937.155288] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 9938.522713] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9938.534753] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9938.625721] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9946.650833] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9950.997553] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9951.231645] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9955.890863] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 9956.083927] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9959.400831] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9959.405654] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9959.418583] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9959.432854] Lustre: Skipped 1 previous similar message [ 9959.450184] Lustre: Skipped 1 previous similar message [ 9963.942267] Lustre: server umount lustre-OST0000 complete [ 9969.121270] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9969.127688] LustreError: Skipped 1 previous similar message [ 9969.137362] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9969.149736] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9973.920452] Lustre: server umount lustre-MDT0000 complete [ 9978.269542] LustreError: 316452:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787403645 with bad export cookie 3559134749455597726 [ 9978.274031] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9978.280072] LustreError: 316452:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9978.741774] Lustre: server umount lustre-MDT0001 complete [ 9982.386993] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 9983.964904] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 09:00:49 (1787403649) [ 9990.472127] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9990.979565] LustreError: 320741:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9990.994833] LustreError: 320741:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 9991.050624] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9995.265808] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [ 9996.257612] LustreError: 320742:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9996.272256] LustreError: 320742:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10001.382413] LustreError: 320741:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10002.567437] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10002.985224] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10007.525887] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [10013.976412] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10019.000814] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10025.417358] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10025.759693] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10031.879895] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [10038.422331] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10044.933750] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10045.147299] Lustre: lustre-OST0001: new disk, initializing [10045.151523] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10045.168091] Lustre: 323586:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10045.296471] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10051.647500] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [10054.323939] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10054.334989] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10054.370432] LustreError: 324046:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10055.432150] LustreError: 324046:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10055.456021] LustreError: 324046:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10056.486683] LustreError: 324051:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10058.610534] LustreError: 324051:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10058.630499] LustreError: 324051:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 4 previous similar messages [10060.492970] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10060.798590] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10062.692381] LustreError: 324046:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10062.707572] LustreError: 324046:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 6 previous similar messages [10066.221186] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff945c86c09000.ost_server_uuid 50 [10067.671460] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff945c86c09000.ost_server_uuid in FULL state after 0 sec [10070.951518] LustreError: 324046:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10070.990894] LustreError: 324046:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [10073.266212] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff945c86c09000.ost_server_uuid 50 [10074.805482] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff945c86c09000.ost_server_uuid in FULL state after 0 sec [10077.229765] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10092.515748] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10092.523858] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10092.537229] LustreError: Skipped 1 previous similar message [10092.547484] Lustre: Skipped 1 previous similar message [10092.548090] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10092.548094] Lustre: Skipped 1 previous similar message [10096.746161] Lustre: server umount lustre-OST0001 complete [10097.972378] LustreError: 324048:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.203.24@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10102.754048] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10102.759523] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10102.774329] Lustre: Skipped 1 previous similar message [10102.779610] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10102.785777] Lustre: Skipped 4 previous similar messages [10105.827175] LustreError: 322575:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10105.848630] LustreError: 322575:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10109.110919] Lustre: server umount lustre-OST0000 complete [10115.554677] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10115.574640] LustreError: Skipped 1 previous similar message [10115.576798] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10115.605966] Lustre: Skipped 1 previous similar message [10115.626344] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10115.642359] Lustre: Skipped 2 previous similar messages [10120.784527] Lustre: server umount lustre-MDT0000 complete [10121.187052] LustreError: 320742:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10121.203204] LustreError: 320742:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10124.846181] LustreError: 320721:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787403792 with bad export cookie 3559134749455598608 [10124.852244] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10124.856708] LustreError: 320721:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10125.277522] Lustre: server umount lustre-MDT0001 complete [10135.602328] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing unload_modules_local [10138.986759] Key type lgssc unregistered [10139.324470] LNet: 326273:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10139.339463] LNetError: 326273:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [10139.373904] LNet: Removed LNI 192.168.203.124@tcp [10140.259342] Key type .llcrypt unregistered [10140.262257] Key type ._llcrypt unregistered [10160.857224] Key type ._llcrypt registered [10160.858955] Key type .llcrypt registered [10160.967753] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [10161.777925] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10161.979988] alg: No test for adler32 (adler32-zlib) [10163.061489] Lustre: Lustre: Build Version: 2.17.54_107_g42be189 [10163.290194] LNet: Added LNI 192.168.203.124@tcp [8/256/0/180] [10164.936164] Key type lgssc registered [10165.881841] Lustre: Echo OBD driver; http://www.lustre.org/ [10172.468829] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10172.497559] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10173.912214] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10177.505832] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [10183.633433] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10184.169801] LustreError: 327676:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10184.201444] LustreError: 327676:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10189.073784] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10189.373795] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10192.954609] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [10202.881295] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10203.318875] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10209.198723] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [10214.585246] LustreError: 329357:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10216.652930] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10219.697838] LustreError: 329356:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10219.725854] LustreError: 329356:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10219.742224] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10223.306691] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10223.646385] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10228.740342] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10229.388508] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [10236.049442] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [10243.354988] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10243.504553] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10346.465177] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10346.472521] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10346.481810] LustreError: Skipped 1 previous similar message [10346.486923] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10346.501165] Lustre: Skipped 1 previous similar message [10346.523167] Lustre: Skipped 1 previous similar message [10350.387094] Lustre: server umount lustre-OST0001 complete [10351.602474] LustreError: 330657:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10351.633683] LustreError: 330657:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10356.710993] LustreError: 329356:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10356.742479] LustreError: 329356:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10358.241282] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10358.248609] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10358.253234] LustreError: Skipped 1 previous similar message [10358.271489] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10361.826150] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10361.840901] Lustre: Skipped 2 previous similar messages [10366.953146] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10366.955136] LustreError: 329357:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10366.956861] Lustre: Skipped 1 previous similar message [10366.987443] LustreError: 329357:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10371.043502] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10371.268983] Lustre: server umount lustre-OST0000 complete [10377.185864] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10377.190038] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10377.208758] Lustre: Skipped 2 previous similar messages [10377.229918] Lustre: Skipped 1 previous similar message [10381.527482] Lustre: server umount lustre-MDT0000 complete [10384.994342] LustreError: 327655:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787404052 with bad export cookie 709748608254787442 [10384.998292] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10385.009492] LustreError: 327655:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10385.370454] Lustre: server umount lustre-MDT0001 complete [10396.291831] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing unload_modules_local [10399.351157] Key type lgssc unregistered [10399.710861] LNet: 332622:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10399.729081] LNetError: 332622:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [10399.750790] LNet: Removed LNI 192.168.203.124@tcp [10400.784243] Key type .llcrypt unregistered [10400.786392] Key type ._llcrypt unregistered [10443.552949] Key type ._llcrypt registered [10443.560969] Key type .llcrypt registered [10443.709301] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [10445.850314] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10445.864672] alg: No test for adler32 (adler32-zlib) [10447.067482] Lustre: Lustre: Build Version: 2.17.54_107_g42be189 [10447.507060] LNet: Added LNI 192.168.203.124@tcp [8/256/0/180] [10449.320525] Key type lgssc registered [10450.675878] Lustre: Echo OBD driver; http://www.lustre.org/ [10460.898131] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10460.927909] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10462.417762] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10467.183886] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [10471.909299] LustreError: 334911:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10471.921600] LustreError: 334911:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10476.417471] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10476.770312] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10481.474361] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [10484.403624] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [10491.218076] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10491.578813] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10497.900945] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [10501.820628] LustreError: 336401:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10506.016799] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10506.331258] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10507.420511] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [10507.424422] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [10512.393366] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [10521.575819] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10529.253619] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [10543.597681] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10543.617489] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10543.618517] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10543.636747] Lustre: Skipped 2 previous similar messages [10546.934830] Lustre: server umount lustre-MDT0000 complete [10548.710061] LustreError: 334908:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10548.730538] LustreError: 334908:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10550.406492] LustreError: 334893:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787404217 with bad export cookie 4366180319475900683 [10550.410731] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10550.421210] LustreError: 334893:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10550.805972] Lustre: server umount lustre-MDT0001 complete [10564.119061] Lustre: server umount lustre-OST0000 complete [10577.719388] Lustre: server umount lustre-OST0001 complete [10593.660605] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [10604.141669] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10604.791330] LustreError: 340180:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10604.925153] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10609.665850] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [10610.155131] LustreError: 340181:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10618.641051] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10619.123721] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10623.114597] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [10626.124656] Lustre: 341321:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10632.578404] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10638.985691] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [10644.274899] LustreError: 341674:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10644.303125] LustreError: 341674:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10647.258777] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10647.561358] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10647.565956] Lustre: Skipped 1 previous similar message [10649.626530] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [10649.653494] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [10653.959993] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [10660.957276] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10663.909430] Lustre: 343193:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [10670.058709] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10670.068893] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10670.076889] Lustre: Skipped 5 previous similar messages [10670.094108] Lustre: Skipped 4 previous similar messages [10673.513230] Lustre: server umount lustre-MDT0000 complete [10675.175277] LustreError: 341696:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10675.187089] LustreError: 341696:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10676.897673] LustreError: 340160:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787404344 with bad export cookie 4366180319475902384 [10676.907619] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10676.914168] LustreError: 340160:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10677.295315] Lustre: server umount lustre-MDT0001 complete [10690.967581] Lustre: server umount lustre-OST0000 complete [10704.625261] Lustre: server umount lustre-OST0001 complete [10708.049644] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [10709.497680] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [10711.108345] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 09:12:56 (1787404376) [10731.501327] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [10739.143904] loop0: detected capacity change from 0 to 1099511627776 [10739.736243] SGI XFS with ACLs, security attributes, quota, no debug enabled [10739.742421] *** VALIDATE xfs *** [10739.749455] XFS (loop0): Mounting V5 Filesystem [10739.780308] XFS (loop0): Ending clean mount [10741.017631] loop1: detected capacity change from 0 to 18691697672192 [10850.797706] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_hostid [10863.348511] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [10896.946132] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10897.376802] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10897.454492] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10897.629439] Lustre: lustre-MDT0000: new disk, initializing [10897.765430] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10897.777701] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10903.544973] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [10917.409289] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10917.520628] Lustre: 348329:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10917.581297] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10917.585306] Lustre: Skipped 1 previous similar message [10917.647382] Lustre: lustre-MDT0001: new disk, initializing [10917.758506] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10917.763967] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10922.842962] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [10933.116940] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10938.979446] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10946.001883] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10946.348727] Lustre: lustre-OST0000: new disk, initializing [10946.360074] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10946.371958] Lustre: 350298:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10947.723296] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10947.733671] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10947.831279] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10953.311116] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [10962.900132] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10967.300919] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10967.527525] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10972.123712] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10972.318430] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10978.786082] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10978.797486] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10978.803803] LustreError: Skipped 1 previous similar message [10978.813415] Lustre: Skipped 1 previous similar message [10978.819888] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10978.830928] Lustre: Skipped 1 previous similar message [10980.475515] Lustre: server umount lustre-OST0000 complete [10984.417785] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10984.430583] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10984.456660] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10989.033180] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10989.045487] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10990.296041] Lustre: server umount lustre-MDT0000 complete [10993.477419] LustreError: 348321:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787404660 with bad export cookie 4366180319475903931 [10993.502370] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10993.793140] Lustre: server umount lustre-MDT0001 complete [11000.737173] XFS (loop0): Unmounting Filesystem [11005.238143] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 09:17:50 (1787404670) [11011.335036] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11011.879784] LustreError: 352854:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11011.978047] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11011.982082] Lustre: Skipped 2 previous similar messages [11015.795656] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [11023.842129] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11028.371568] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [11035.211575] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11040.956241] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11047.176778] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11053.595134] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [11061.388217] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11073.510901] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11073.510919] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11073.514578] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11073.543853] Lustre: Skipped 2 previous similar messages [11076.773947] Lustre: server umount lustre-OST0000 complete [11080.672940] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11086.968933] Lustre: server umount lustre-MDT0000 complete [11088.866660] LustreError: 352855:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11088.878232] LustreError: 352855:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11090.457055] LustreError: 352836:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787404757 with bad export cookie 4366180319475904820 [11090.461983] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11090.470377] LustreError: 352836:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [11090.770988] Lustre: server umount lustre-MDT0001 complete [11100.217856] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing unload_modules_local [11102.678765] Key type lgssc unregistered [11102.995720] LNet: 356484:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11103.004058] LNetError: 356484:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [11103.027058] LNet: Removed LNI 192.168.203.124@tcp [11103.852230] Key type .llcrypt unregistered [11103.860754] Key type ._llcrypt unregistered [11131.732405] Key type ._llcrypt registered [11131.735676] Key type .llcrypt registered [11131.923637] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [11132.961935] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11133.007318] alg: No test for adler32 (adler32-zlib) [11134.031715] Lustre: Lustre: Build Version: 2.17.54_107_g42be189 [11134.171734] LNet: Added LNI 192.168.203.124@tcp [8/256/0/180] [11135.832389] Key type lgssc registered [11137.035520] Lustre: Echo OBD driver; http://www.lustre.org/ [11145.287874] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11145.313081] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11146.808098] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11151.846763] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [11155.949058] LustreError: 358050:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11156.001888] LustreError: 358050:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11161.075952] LustreError: 358049:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11163.599745] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11163.906117] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11167.945848] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [11175.818808] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11181.331114] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11188.639610] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11189.276866] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11198.360685] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [11209.959863] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11220.456378] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11220.477522] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11220.514520] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11221.853898] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.24@tcp (stopping) [11221.988280] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11223.322609] Lustre: server umount lustre-MDT0001 complete [11226.925522] LustreError: 358046:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.24@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11232.050963] LustreError: 358046:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.24@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11232.079801] LustreError: 358046:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11235.853077] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11236.007701] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11236.022334] Lustre: Found index 1 for lustre-MDT0001, updating log [11236.094049] Lustre: 358037:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11236.483900] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11240.952413] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [11241.983533] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11243.012608] LustreError: 361616:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11243.025463] LustreError: 361616:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11243.033945] LustreError: 358041:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11243.105770] LustreError: 361624:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.124@tcp: cfg command failed: rc = -17 [11243.116958] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11243.116958] [11252.197068] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11252.221166] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11252.222256] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11252.228567] Lustre: Skipped 3 previous similar messages [11256.466974] Lustre: server umount lustre-MDT0001 complete [11257.313649] LustreError: 358049:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11257.326995] LustreError: 358049:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [11266.710034] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11266.880283] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11266.888267] Lustre: Found index 1 for lustre-MDT0001, updating log [11266.934244] Lustre: 358038:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11267.193362] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11271.923325] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [11272.697669] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11274.250699] LustreError: 362700:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11274.256745] LustreError: 362700:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11274.266666] LustreError: 358041:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11274.272221] LustreError: 358041:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11274.352917] LustreError: 362709:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [11274.365990] LustreError: 362709:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [11274.372417] LustreError: 362709:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.124@tcp: cfg command failed: rc = -17 [11274.378499] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.203.124@tcp [11274.378499] [11579.875891] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11579.893657] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11579.907508] Lustre: Skipped 1 previous similar message [11579.914617] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11579.920649] Lustre: Skipped 1 previous similar message [11582.880300] Lustre: server umount lustre-MDT0001 complete [11584.995204] LustreError: 358050:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11585.025309] LustreError: 358050:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [11593.681857] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11593.761555] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11593.765677] Lustre: Found index 1 for lustre-MDT0001, updating log [11593.799875] Lustre: 358037:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11594.102767] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [11598.333105] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [11599.338656] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11603.451974] LustreError: 371612:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11603.466563] LustreError: 371612:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11603.480456] LustreError: 358041:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11603.486409] LustreError: 358041:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11603.565329] LustreError: 371615:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [11603.575269] LustreError: 371615:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.124@tcp: cfg command failed: rc = -114 [11603.598632] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [11603.598632] [11921.892267] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11921.899226] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11921.904377] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11921.904385] Lustre: Skipped 1 previous similar message [11921.942332] Lustre: Skipped 3 previous similar messages [11932.131811] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11932.138321] Lustre: Skipped 7 previous similar messages [11934.176351] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11934.376737] Lustre: server umount lustre-MDT0000 complete [11937.253070] LustreError: 358044:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11937.297273] LustreError: 358044:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [11938.409325] LustreError: 359894:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787405605 with bad export cookie 12803512224687813445 [11938.410966] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11938.434176] LustreError: 359894:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11942.380428] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11945.132242] Lustre: server umount lustre-MDT0001 complete [11949.549160] Lustre: server umount lustre-OST0000 complete [11958.860993] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_hostid [11969.760680] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [12005.884745] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 09:34:31 (1787405671) [12011.980771] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12012.219549] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12012.258423] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12012.348811] Lustre: lustre-MDT0000: new disk, initializing [12012.439912] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12012.456217] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12016.071520] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [12026.080832] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12026.227830] Lustre: 382432:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12026.306071] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12026.310274] Lustre: Skipped 1 previous similar message [12026.448222] Lustre: lustre-MDT0001: new disk, initializing [12026.563737] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12026.639140] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12026.648672] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12031.067754] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [12040.523866] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [12046.169231] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [12053.879426] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12054.417300] Lustre: lustre-OST0000: new disk, initializing [12054.426332] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12054.438788] Lustre: 384398:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12054.590969] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12061.871147] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_default_debug -1 all [12062.347583] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12062.371715] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12062.556134] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12072.152577] Lustre: DEBUG MARKER: oleg324-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [12081.135962] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [12081.141744] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12081.156441] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12081.167826] Lustre: Skipped 2 previous similar messages [12085.364681] Lustre: server umount lustre-MDT0000 complete [12088.290934] LustreError: 382438:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12088.317045] LustreError: 382438:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [12089.297449] LustreError: 382424:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787405756 with bad export cookie 12803512224687815398 [12089.308567] LustreError: MGC192.168.203.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12089.309181] LustreError: 383239:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [12089.309409] LustreError: 383239:0:(client.c:1381:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@000000002daa09b9 x1874229776919424/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 2 fl Rpc:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [12089.309871] LustreError: 382424:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12089.696604] Lustre: server umount lustre-MDT0001 complete [12103.986934] Lustre: server umount lustre-OST0000 complete [12134.666360] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing set_hostid [12145.092339] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing load_modules_local [12173.007616] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 11923 sec ========= 09:37:18 (1787405838) [12174.623719] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 09:37:20 (1787405840) === [12177.791675] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 09:37:23 (1787405843) === [12213.368912] Lustre: DEBUG MARKER: oleg324-server.virtnet: executing unload_modules_local [12216.279947] Key type lgssc unregistered [12216.561816] LNet: 390765:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12216.569447] LNetError: 390765:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [12216.590674] LNet: Removed LNI 192.168.203.124@tcp [12217.496202] Key type .llcrypt unregistered [12217.498141] Key type ._llcrypt unregistered