[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 491932493 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.003161] x2apic enabled [ 0.004009] Switched APIC routing to physical x2apic. [ 0.005013] kvm-guest: setup PV IPIs [ 0.008482] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009021] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.011013] pid_max: default: 32768 minimum: 301 [ 0.012143] LSM: Security Framework initializing [ 0.013047] Yama: becoming mindful. [ 0.014042] SELinux: Initializing. [ 0.015080] *** VALIDATE selinux *** [ 0.024342] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.029120] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.030159] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031109] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.032136] *** VALIDATE tmpfs *** [ 0.033512] *** VALIDATE proc *** [ 0.035204] *** VALIDATE cgroup *** [ 0.036015] *** VALIDATE cgroup2 *** [ 0.038096] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.039166] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.040012] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.041031] Spectre V2 : User space: Vulnerable [ 0.042010] Speculative Store Bypass: Vulnerable [ 0.045564] debug: unmapping init [mem 0xffffffffad859000-0xffffffffad860fff] [ 0.047192] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.048734] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.049026] ... version: 2 [ 0.050012] ... bit width: 48 [ 0.051013] ... generic registers: 4 [ 0.052014] ... value mask: 0000ffffffffffff [ 0.053016] ... max period: 00007fffffffffff [ 0.054017] ... fixed-purpose events: 3 [ 0.055012] ... event mask: 000000070000000f [ 0.056327] rcu: Hierarchical SRCU implementation. [ 0.058421] smp: Bringing up secondary CPUs ... [ 0.059643] x86: Booting SMP configuration: [ 0.060032] .... node #0, CPUs: #1 #2 #3 [ 0.064012] smp: Brought up 1 node, 4 CPUs [ 0.066017] smpboot: Max logical packages: 1 [ 0.067012] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.144025] node 0 deferred pages initialised in 74ms [ 0.149010] devtmpfs: initialized [ 0.150329] x86/mm: Memory block size: 128MB [ 0.153513] gcov: version magic: 0x41383552 [ 0.157255] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.161124] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.165349] pinctrl core: initialized pinctrl subsystem [ 0.168211] [ 0.168948] ************************************************************* [ 0.172016] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.175014] ** ** [ 0.178027] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.180023] ** ** [ 0.183027] ** This means that this kernel is built to expose internal ** [ 0.186027] ** IOMMU data structures, which may compromise security on ** [ 0.188022] ** your system. ** [ 0.189020] ** ** [ 0.191024] ** If you see this message and you are not debugging the ** [ 0.194026] ** kernel, report this immediately to your vendor! ** [ 0.197024] ** ** [ 0.200027] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.202020] ************************************************************* [ 0.205929] NET: Registered protocol family 16 [ 0.207622] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.211145] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.214168] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.218203] cpuidle: using governor menu [ 0.220425] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.224084] PCI: Using configuration type 1 for base access [ 0.226163] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.236180] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.237093] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.239041] cryptd: max_cpu_qlen set to 1000 [ 0.241280] ACPI: Added _OSI(Module Device) [ 0.242015] ACPI: Added _OSI(Processor Device) [ 0.243016] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.244047] ACPI: Added _OSI(Processor Aggregator Device) [ 0.249427] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.256345] ACPI: Interpreter enabled [ 0.258131] ACPI: PM: (supports S0 S3 S4 S5) [ 0.260022] ACPI: Using IOAPIC for interrupt routing [ 0.262166] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.266606] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.279303] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.282070] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.285036] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.289159] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.295011] acpiphp: Slot [2] registered [ 0.297252] acpiphp: Slot [5] registered [ 0.299272] acpiphp: Slot [6] registered [ 0.300355] acpiphp: Slot [7] registered [ 0.303200] acpiphp: Slot [8] registered [ 0.304221] acpiphp: Slot [9] registered [ 0.306241] acpiphp: Slot [10] registered [ 0.308208] acpiphp: Slot [3] registered [ 0.310189] acpiphp: Slot [4] registered [ 0.311167] acpiphp: Slot [11] registered [ 0.313164] acpiphp: Slot [12] registered [ 0.315189] acpiphp: Slot [13] registered [ 0.317121] acpiphp: Slot [14] registered [ 0.318124] acpiphp: Slot [15] registered [ 0.320104] acpiphp: Slot [16] registered [ 0.321114] acpiphp: Slot [17] registered [ 0.323095] acpiphp: Slot [18] registered [ 0.324117] acpiphp: Slot [19] registered [ 0.325118] acpiphp: Slot [20] registered [ 0.327096] acpiphp: Slot [21] registered [ 0.328109] acpiphp: Slot [22] registered [ 0.330121] acpiphp: Slot [23] registered [ 0.331095] acpiphp: Slot [24] registered [ 0.333120] acpiphp: Slot [25] registered [ 0.334134] acpiphp: Slot [26] registered [ 0.336125] acpiphp: Slot [27] registered [ 0.338129] acpiphp: Slot [28] registered [ 0.339141] acpiphp: Slot [29] registered [ 0.341165] acpiphp: Slot [30] registered [ 0.342136] acpiphp: Slot [31] registered [ 0.344072] PCI host bridge to bus 0000:00 [ 0.345021] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.347054] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.350043] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.353046] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.356039] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.357045] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.359208] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.362196] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.366436] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.376856] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.381950] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.385036] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.387043] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.390033] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.394157] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.395690] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.398035] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.399740] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.405016] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.416014] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.420015] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.426594] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.435027] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.443024] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.459000] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.470000] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.477031] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.486016] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.502028] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.512000] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.519014] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.529017] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.547018] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.558022] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.565030] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.574035] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.601031] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.621959] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.629022] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.635020] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.651020] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.661898] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.685024] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.701024] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.737014] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.755000] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.757455] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.760448] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.763394] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.766288] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.773032] iommu: Default domain type: Passthrough [ 0.774418] SCSI subsystem initialized [ 0.775145] ACPI: bus type USB registered [ 0.776121] usbcore: registered new interface driver usbfs [ 0.778102] usbcore: registered new interface driver hub [ 0.780073] usbcore: registered new device driver usb [ 0.782164] pps_core: LinuxPPS API ver. 1 registered [ 0.784012] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.787061] PTP clock support registered [ 0.790038] EDAC MC: Ver: 3.0.0 [ 0.791463] PCI: Using ACPI for IRQ routing [ 0.792788] NetLabel: Initializing [ 0.793008] NetLabel: domain hash size = 128 [ 0.794010] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.795218] NetLabel: unlabeled traffic allowed by default [ 0.797183] vgaarb: loaded [ 0.799277] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.801013] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.807899] clocksource: Switched to clocksource kvm-clock [ 0.915674] VFS: Disk quotas dquot_6.6.0 [ 0.917239] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.920167] *** VALIDATE ramfs *** [ 0.921576] *** VALIDATE hugetlbfs *** [ 0.923186] pnp: PnP ACPI init [ 0.926298] pnp: PnP ACPI: found 6 devices [ 0.981399] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.985945] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.989122] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.992165] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.995714] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.999049] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.003063] NET: Registered protocol family 2 [ 1.006261] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.011372] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.015095] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.021352] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.025800] TCP: Hash tables configured (established 65536 bind 65536) [ 1.029358] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.033270] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.036373] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.039609] NET: Registered protocol family 1 [ 1.042618] RPC: Registered named UNIX socket transport module. [ 1.047066] RPC: Registered udp transport module. [ 1.049107] RPC: Registered tcp transport module. [ 1.051125] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.053986] NET: Registered protocol family 44 [ 1.056670] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.060354] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.063956] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.067339] PCI: CLS 0 bytes, default 64 [ 1.069351] Unpacking initramfs... [ 2.501132] debug: unmapping init [mem 0xffff92c2fcc54000-0xffff92c2fffbffff] [ 2.506762] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.510083] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.514414] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.026681] Initialise system trusted keyrings [ 3.028661] Key type blacklist registered [ 3.031108] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.042273] zbud: loaded [ 3.045804] *** VALIDATE nfs *** [ 3.047369] *** VALIDATE nfs4 *** [ 3.049471] pstore: using deflate compression [ 3.053532] Platform Keyring initialized [ 3.166911] NET: Registered protocol family 38 [ 3.169421] Key type asymmetric registered [ 3.171401] Asymmetric key parser 'x509' registered [ 3.173984] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.177202] io scheduler mq-deadline registered [ 3.178989] io scheduler kyber registered [ 3.180846] io scheduler bfq registered [ 3.182731] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.186264] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.189295] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.192443] ACPI: Power Button [PWRF] [ 3.199289] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.207266] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.234510] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.242112] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.264765] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.294580] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.324257] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.330853] Non-volatile memory driver v1.3 [ 3.332765] Linux agpgart interface v0.103 [ 3.365843] virtio_blk virtio1: [vda] 146960 512-byte logical blocks (75.2 MB/71.8 MiB) [ 3.368984] vda: detected capacity change from 0 to 75243520 [ 3.383869] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.387237] vdb: detected capacity change from 0 to 1073741824 [ 3.417349] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.420096] vdc: detected capacity change from 0 to 2621440000 [ 3.437470] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.440700] vdd: detected capacity change from 0 to 2621440000 [ 3.460150] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.463521] vde: detected capacity change from 0 to 4294967296 [ 3.482307] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.485844] vdf: detected capacity change from 0 to 4294967296 [ 3.493293] libphy: Fixed MDIO Bus: probed [ 3.498981] usbcore: registered new interface driver usbserial_generic [ 3.501888] usbserial: USB Serial support registered for generic [ 3.504606] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.510423] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.512826] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.515835] mousedev: PS/2 mouse device common for all mice [ 3.521459] rtc_cmos 00:05: RTC can wake from S4 [ 3.524866] rtc_cmos 00:05: registered as rtc0 [ 3.525118] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.527050] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.537359] intel_pstate: CPU model not supported [ 3.539658] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.548663] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.549212] hid: raw HID events driver (C) Jiri Kosina [ 3.554554] usbcore: registered new interface driver usbhid [ 3.556817] usbhid: USB HID core driver [ 3.559069] drop_monitor: Initializing network drop monitor service [ 3.564228] Initializing XFRM netlink socket [ 3.566898] NET: Registered protocol family 10 [ 3.570564] Segment Routing with IPv6 [ 3.572201] NET: Registered protocol family 17 [ 3.574128] mpls_gso: MPLS GSO support [ 3.581069] RAS: Correctable Errors collector initialized. [ 3.583184] AVX version of gcm_enc/dec engaged. [ 3.585093] AES CTR mode by8 optimization enabled [ 3.692167] sched_clock: Marking stable (3692014664, 0)->(4622717735, -930703071) [ 3.696258] registered taskstats version 1 [ 3.699439] Loading compiled-in X.509 certificates [ 3.701964] zswap: loaded using pool lzo/zbud [ 3.736680] Key type big_key registered [ 3.751451] Key type encrypted registered [ 3.753171] ima: No TPM chip found, activating TPM-bypass! [ 3.755479] ima: Allocated hash algorithm: sha1 [ 3.757159] ima: No architecture policies found [ 3.759156] evm: Initialising EVM extended attributes: [ 3.761223] evm: security.selinux [ 3.762375] evm: security.ima [ 3.763846] evm: security.capability [ 3.765212] evm: HMAC attrs: 0x1 [ 3.767884] rtc_cmos 00:05: setting system clock to 2026-08-27 22:07:35 UTC (1787868455) [ 3.774693] debug: unmapping init [mem 0xffffffffae803000-0xffffffffae9fffff] [ 3.777947] debug: unmapping init [mem 0xffffffffad582000-0xffffffffad858fff] [ 3.786137] Write protecting the kernel read-only data: 28672k [ 3.789945] debug: unmapping init [mem 0xffffffffabc03000-0xffffffffabdfffff] [ 3.793144] debug: unmapping init [mem 0xffffffffac514000-0xffffffffac5fffff] [ 3.836485] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.848085] systemd[1]: Detected virtualization kvm. [ 3.850457] systemd[1]: Detected architecture x86-64. [ 3.853032] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.884471] systemd[1]: No hostname configured. [ 3.886625] systemd[1]: Set hostname to . [ 3.889148] random: systemd: uninitialized urandom read (16 bytes read) [ 3.892186] systemd[1]: Initializing machine ID from random generator. [ 3.942989] random: ln: uninitialized urandom read (6 bytes read) [ 4.030061] random: systemd: uninitialized urandom read (16 bytes read) [ 4.033386] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ 4.039726] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ 4.045162] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ OK ] Reached target Swap. [ OK ] Listening on udev Control Socket. [ OK ] Listening on Journal Socket. Starting Setup Virtual Console... Starting Journal Service... Starting Apply Kernel Variables... [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Slices. [ OK ] Reached target Sockets. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Reached target Paths. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Journal Service. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.762710] device-mapper: uevent: version 1.0.3 [ 4.765517] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook.[ 5.192157] random: fast init done Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.632802] virtio_net virtio0 ens2: renamed from eth0 [ 5.761484] scsi host0: ata_piix [ 5.817744] scsi host1: ata_piix [ 5.831753] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.834735] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.433600] dracut-initqueue[581]: RTNETLINK answers: File exists [ 10.271599] random: crng init done [ 10.273133] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.788876] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Timers. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local File Systems. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.927931] printk: systemd: 23 output lines suppressed due to ratelimiting [ 12.200166] SELinux: Disabled at runtime. [ 12.262610] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.272393] systemd[1]: Detected virtualization kvm. [ 12.274179] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.794382] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.799130] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.803418] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.807871] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.813821] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.826615] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.844368] systemd[1]: Listening on Process Core Dump Socket. [ OK ] Listening on Process Core Dump Socket. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target rpc_pipefs.target. [ OK ] Created slice system-serial\x2dgetty.slice. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on udev Kernel Socket. Activating swap /dev/disk/by-label/SWAP... Mounting Kernel Debug File System... Mounting POSIX Message Queue File System... Mounting Huge Pages File System... [ 12.913762] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Created slice system-getty.slice. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. Starting Remount Root and Kernel File Systems... [ OK ] Stopped target Initrd File Systems. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. Starting Apply Kernel Variables... [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Started Journal Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Huge Pages File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Apply Kernel Variables. Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 13.264114] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.581827] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.610519] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.732866] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.768217] EDAC sbridge: Ver: 1.1.2 [ 15.213296] Key type dns_resolver registered [ 15.517180] NFS: Registering the id_resolver key type [ 15.521662] Key type id_resolver registered [ 15.523040] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Login Service... Starting Restore /run/initramfs on shutdown... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started irqbalance daemon. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg448-server login: [ 28.348139] hrtimer: interrupt took 3129420 ns [ 76.715150] libcfs: loading out-of-tree module taints kernel. [ 76.851483] Key type ._llcrypt registered [ 76.854158] Key type .llcrypt registered [ 77.107931] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_hostid [ 100.476588] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 101.913690] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 101.926405] alg: No test for adler32 (adler32-zlib) [ 103.282796] Lustre: Lustre: Build Version: 2.17.57_99_gb2219ab [ 104.306258] LNet: Added LNI 192.168.204.148@tcp [8/256/0/180] [ 106.167686] Key type lgssc registered [ 108.308976] Lustre: Echo OBD driver; http://www.lustre.org/ [ 128.333410] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 174.078964] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 189.292289] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 189.326974] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 190.557442] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 190.593913] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 190.710228] Lustre: lustre-MDT0000: new disk, initializing [ 190.864852] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 190.912593] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 196.548556] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 211.776692] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 211.868713] Lustre: 6516:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 211.911783] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 211.917760] Lustre: Skipped 1 previous similar message [ 211.995992] Lustre: lustre-MDT0001: new disk, initializing [ 212.069402] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 212.111314] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 212.134785] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 216.802544] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 221.999089] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 234.365661] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 234.709802] Lustre: lustre-OST0000: new disk, initializing [ 234.713228] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 234.719830] Lustre: 8454:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 234.774101] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 236.759647] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 236.772609] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 236.927185] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 242.015133] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 260.064136] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 260.170669] Lustre: lustre-OST0001: new disk, initializing [ 260.174545] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 260.182851] Lustre: 9526:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 260.247332] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 266.959620] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 267.347082] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 267.365403] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 267.463862] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 281.171169] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 288.830559] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 296.188811] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing check_logdir /tmp/testlogs/ [ 301.786884] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing yml_node [ 306.358420] Lustre: DEBUG MARKER: Client: 2.17.57.99 [ 309.090287] Lustre: DEBUG MARKER: MDS: 2.17.57.99 [ 312.256585] Lustre: DEBUG MARKER: OSS: 2.17.57.99 [ 314.935443] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Thu Aug 27 18:12:44 EDT 2026 [ 333.899350] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 335.347616] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 354.784676] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 354.786298] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 354.801415] Lustre: Skipped 3 previous similar messages [ 354.808545] Lustre: Skipped 3 previous similar messages [ 358.155794] Lustre: server umount lustre-MDT0000 complete [ 362.896580] LustreError: 6507:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787868814 with bad export cookie 5812480057813352669 [ 362.901696] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 362.909840] LustreError: 6507:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 365.024544] LustreError: 6523:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 365.025247] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 365.051544] LustreError: 6523:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 6 previous similar messages [ 365.085605] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 365.134745] Lustre: Skipped 1 previous similar message [ 367.073374] LustreError: 6522:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 367.073705] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 367.119881] LustreError: 6522:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 367.125188] Lustre: Skipped 1 previous similar message [ 372.192366] LustreError: 12468:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 372.192509] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 372.214868] LustreError: 12468:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 372.220722] Lustre: Skipped 1 previous similar message [ 377.311557] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 377.714631] Lustre: server umount lustre-MDT0001 complete [ 381.784754] Lustre: server umount lustre-OST0000 complete [ 385.922953] Lustre: server umount lustre-OST0001 complete [ 392.846838] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_hostid [ 403.755577] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 444.062938] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 444.282319] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 444.308487] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 444.385649] Lustre: lustre-MDT0000: new disk, initializing [ 444.450672] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 444.488623] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 448.740251] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 459.640533] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 459.786245] Lustre: 15078:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 459.833207] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 459.836973] Lustre: Skipped 1 previous similar message [ 459.924298] Lustre: lustre-MDT0001: new disk, initializing [ 460.010796] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 460.062811] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 460.073167] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 466.026218] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 476.341188] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 483.065455] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 491.511932] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 492.013792] Lustre: lustre-OST0000: new disk, initializing [ 492.029064] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 492.054274] Lustre: 17048:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 492.297504] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 493.875516] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 493.895710] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 494.106150] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 499.265329] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 510.549663] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 516.591431] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 516.820872] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 522.102661] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 522.356675] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 524.785101] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 524.788782] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 524.799732] LustreError: Skipped 1 previous similar message [ 524.802096] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 524.802108] Lustre: Skipped 2 previous similar messages [ 524.853029] Lustre: Skipped 2 previous similar messages [ 530.918298] Lustre: server umount lustre-OST0000 complete [ 537.060341] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 537.069894] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 537.098575] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 537.112692] Lustre: Skipped 3 previous similar messages [ 541.777861] Lustre: server umount lustre-MDT0000 complete [ 545.248568] LustreError: 15092:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 545.271979] LustreError: 15092:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [ 545.427710] LustreError: 15072:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787868997 with bad export cookie 5812480057813354202 [ 545.440978] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 545.442595] LustreError: 15072:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 545.825887] Lustre: server umount lustre-MDT0001 complete [ 551.692174] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 18:16:41 (1787869001) [ 559.444568] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 559.859197] LustreError: 19316:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 559.953388] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 564.720772] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 569.330135] LustreError: 19317:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 569.361182] LustreError: 19317:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 574.517119] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 580.762773] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 589.715645] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 596.829900] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 604.566257] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 604.647738] LustreError: 21136:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 604.670653] LustreError: 21136:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 611.044556] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 618.162408] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 625.264937] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 634.852854] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 634.857596] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 634.866233] Lustre: Skipped 3 previous similar messages [ 634.886272] Lustre: Skipped 4 previous similar messages [ 638.098299] Lustre: server umount lustre-MDT0000 complete [ 639.970545] LustreError: 19316:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 639.988780] LustreError: 19316:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 641.776094] LustreError: 19296:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787869093 with bad export cookie 5812480057813355084 [ 641.792364] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 642.124337] Lustre: server umount lustre-MDT0001 complete [ 656.812557] Lustre: server umount lustre-OST0000 complete [ 678.864511] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 690.364082] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 690.920194] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 690.927747] Lustre: Skipped 2 previous similar messages [ 695.724906] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 704.384288] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 710.218173] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 713.469303] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 721.472287] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 729.343698] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 738.907047] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 739.111748] Lustre: lustre-OST0001: new disk, initializing [ 739.115188] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 739.121181] Lustre: 26017:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 740.450559] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 740.455965] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 740.544964] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 746.293310] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 756.903474] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 761.395908] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 776.687089] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 776.687318] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 776.692619] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 776.730337] Lustre: Skipped 2 previous similar messages [ 781.306841] Lustre: server umount lustre-MDT0000 complete [ 781.793899] LustreError: 24433:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 781.823200] LustreError: 24433:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 6 previous similar messages [ 785.191729] LustreError: 26784:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787869236 with bad export cookie 5812480057813355931 [ 785.197392] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 785.213786] LustreError: 26784:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 785.682397] Lustre: server umount lustre-MDT0001 complete [ 801.036206] Lustre: server umount lustre-OST0000 complete [ 802.912712] Lustre: 3649:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787869238/real 1787869238] req@ffff92c37a99cf00 x1874716059227264/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787869254 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 802.938682] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 802.948319] Lustre: Skipped 1 previous similar message [ 805.223698] Lustre: server umount lustre-OST0001 complete [ 822.828725] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 836.916313] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 837.651837] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 837.671908] Lustre: Skipped 3 previous similar messages [ 843.294466] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 853.850398] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 859.477849] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 862.924949] Lustre: 30189:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 869.934155] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 877.200449] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 886.436249] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 893.610824] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 902.242487] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 906.532090] Lustre: 32070:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 915.940123] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 915.965647] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 915.980284] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 915.990882] Lustre: Skipped 3 previous similar messages [ 920.365791] Lustre: server umount lustre-MDT0000 complete [ 922.607904] LustreError: 29041:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 922.643518] LustreError: 29041:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 10 previous similar messages [ 923.749386] LustreError: 29022:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787869375 with bad export cookie 5812480057813357226 [ 923.757675] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 923.771197] LustreError: 29022:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 924.085283] Lustre: server umount lustre-MDT0001 complete [ 939.008764] Lustre: server umount lustre-OST0000 complete [ 952.644183] Lustre: server umount lustre-OST0001 complete [ 964.070316] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 18:23:33 (1787869413) [ 1000.899533] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 1011.034154] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1016.022549] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 1025.513412] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1030.831784] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 1033.650898] Lustre: 36026:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1040.730022] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1047.507788] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 1058.045491] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1065.544683] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 1073.900401] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1077.758539] Lustre: 37905:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1083.885558] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1083.890363] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1083.923442] Lustre: Skipped 6 previous similar messages [ 1083.937659] Lustre: Skipped 7 previous similar messages [ 1090.249435] Lustre: server umount lustre-MDT0000 complete [ 1094.801307] LustreError: 34859:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787869546 with bad export cookie 5812480057813358332 [ 1094.809397] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1095.066532] Lustre: server umount lustre-MDT0001 complete [ 1111.817723] Lustre: server umount lustre-OST0000 complete [ 1113.059052] Lustre: 3647:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787869548/real 1787869548] req@ffff92c2443a6940 x1874716059332352/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787869564 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1114.594589] Lustre: 3649:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787869550/real 1787869550] req@ffff92c3617821c0 x1874716059332608/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787869566 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1116.551541] Lustre: server umount lustre-OST0001 complete [ 1128.704583] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 18:26:18 (1787869578) [ 1153.957824] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 1166.243552] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1166.914345] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1166.919956] Lustre: Skipped 7 previous similar messages [ 1172.083173] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 1182.180914] LustreError: 40075:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1182.200315] LustreError: 40075:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 14 previous similar messages [ 1182.755296] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1189.149189] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 1192.643225] Lustre: 41226:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1200.996589] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1208.407475] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 1220.559897] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1230.611451] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 1241.014382] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1245.615201] Lustre: 43106:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1246.821333] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1246.828674] Lustre: Skipped 1 previous similar message [ 1251.815797] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1251.844743] Lustre: Skipped 3 previous similar messages [ 1256.584690] Lustre: server umount lustre-MDT0000 complete [ 1261.088158] LustreError: 40056:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787869712 with bad export cookie 5812480057813359438 [ 1261.090353] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1261.099578] LustreError: 40056:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 1267.700540] Lustre: server umount lustre-MDT0001 complete [ 1278.792644] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1291.718956] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 1295.346543] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1300.805318] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1306.166664] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 1306.607393] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1306.621547] Lustre: Skipped 1 previous similar message [ 1314.527940] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1316.393922] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1323.022721] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1324.971969] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1332.197141] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1337.837308] Lustre: server umount lustre-MDT0000 complete [ 1342.494664] LustreError: 40055:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787869794 with bad export cookie 5812480057813360586 [ 1342.508615] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1342.518420] LustreError: 40055:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1343.026735] Lustre: server umount lustre-MDT0001 complete [ 1357.895909] Lustre: server umount lustre-OST0000 complete [ 1371.786682] Lustre: server umount lustre-OST0001 complete [ 1383.182755] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 18:30:33 (1787869833) [ 1402.475991] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 1414.749850] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1420.177200] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 1430.385586] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1437.054862] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 1441.078400] Lustre: 48872:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1450.276260] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1458.977331] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 1470.835731] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1479.988784] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 1489.365669] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1495.182466] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1512.425645] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1512.438686] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1512.453566] Lustre: Skipped 9 previous similar messages [ 1512.474037] Lustre: Skipped 26 previous similar messages [ 1515.943052] Lustre: server umount lustre-MDT0000 complete [ 1519.995970] LustreError: 50757:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787869971 with bad export cookie 5812480057813361412 [ 1520.004719] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1520.023359] LustreError: 50757:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1520.532424] Lustre: server umount lustre-MDT0001 complete [ 1535.170697] Lustre: server umount lustre-OST0000 complete [ 1538.850122] Lustre: 3647:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787869974/real 1787869974] req@ffff92c37c452d00 x1874716059496320/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787869990 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1539.694818] Lustre: server umount lustre-OST0001 complete [ 1549.554726] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 18:33:19 (1787869999) [ 1552.530973] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1554.890793] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 18:33:25 (1787870005) [ 1568.590365] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_hostid [ 1580.150507] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 1606.924040] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1607.313490] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1607.439192] Lustre: lustre-MDT0000: new disk, initializing [ 1607.590571] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1613.277616] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 1624.181687] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1632.721451] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1633.016674] Lustre: lustre-OST0000: new disk, initializing [ 1633.022732] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1633.032694] Lustre: Skipped 1 previous similar message [ 1633.037337] Lustre: 54956:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1634.705124] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1634.713887] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1634.765217] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1640.472420] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 1651.810591] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1654.616618] Lustre: DEBUG MARKER: create test files [ 1701.345302] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1703.928082] Lustre: server umount lustre-OST0000 complete [ 1709.740156] Lustre: server umount lustre-MDT0000 complete [ 1727.266628] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing unload_modules_local [ 1730.634932] Key type lgssc unregistered [ 1731.043958] LNet: 56594:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1731.052883] LNetError: 56594:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1731.065892] LNet: Removed LNI 192.168.204.148@tcp [ 1732.125188] Key type .llcrypt unregistered [ 1732.128849] Key type ._llcrypt unregistered [ 1767.657915] Key type ._llcrypt registered [ 1767.669168] Key type .llcrypt registered [ 1767.803142] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 1769.548996] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1769.579577] alg: No test for adler32 (adler32-zlib) [ 1770.836738] Lustre: Lustre: Build Version: 2.17.57_99_gb2219ab [ 1771.109371] LNet: Added LNI 192.168.204.148@tcp [8/256/0/180] [ 1772.807176] Key type lgssc registered [ 1774.175054] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1784.336985] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1784.363024] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1786.028820] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1791.679281] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 1801.398332] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1810.726343] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1811.376811] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1816.576166] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1816.740952] Lustre: 59331:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1816.762482] Lustre: 59331:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1816.776886] Lustre: 59331:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1816.796604] Lustre: 59331:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1816.813972] Lustre: 59331:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1816.825206] Lustre: 59331:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1819.804133] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 1829.042514] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1835.097747] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1842.416933] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1850.529882] Lustre: server umount lustre-OST0000 complete [ 1865.436635] Lustre: server umount lustre-MDT0000 complete [ 1879.572828] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing unload_modules_local [ 1882.722026] Key type lgssc unregistered [ 1883.036733] LNet: 60944:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1883.064504] LNetError: 60944:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1883.085373] LNet: Removed LNI 192.168.204.148@tcp [ 1884.128903] Key type .llcrypt unregistered [ 1884.133582] Key type ._llcrypt unregistered [ 1919.071347] Key type ._llcrypt registered [ 1919.081757] Key type .llcrypt registered [ 1919.225644] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 1920.621023] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1920.706923] alg: No test for adler32 (adler32-zlib) [ 1921.834268] Lustre: Lustre: Build Version: 2.17.57_99_gb2219ab [ 1922.184268] LNet: Added LNI 192.168.204.148@tcp [8/256/0/180] [ 1923.863399] Key type lgssc registered [ 1925.362801] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1934.785460] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1934.810525] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1936.242283] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1941.893600] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 1950.217857] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1957.967560] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1958.405822] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1960.523841] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1960.578703] Lustre: 63784:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1960.590434] Lustre: 63784:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1960.599205] Lustre: 63784:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1960.606980] Lustre: 63784:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1960.615542] Lustre: 63784:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1960.626417] Lustre: 63784:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1965.349684] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 1974.291551] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1980.277933] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 2021.856739] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2021.872152] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2021.890665] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2025.657336] Lustre: server umount lustre-OST0000 complete [ 2030.535190] Lustre: server umount lustre-MDT0000 complete [ 2043.310307] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing unload_modules_local [ 2046.992481] Key type lgssc unregistered [ 2047.264230] LNet: 65376:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2047.278533] LNetError: 65376:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2047.290872] LNet: Removed LNI 192.168.204.148@tcp [ 2048.371335] Key type .llcrypt unregistered [ 2048.374074] Key type ._llcrypt unregistered [ 2071.572324] Key type ._llcrypt registered [ 2071.585559] Key type .llcrypt registered [ 2071.794265] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_hostid [ 2094.535559] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 2095.744635] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2095.803966] alg: No test for adler32 (adler32-zlib) [ 2096.990887] Lustre: Lustre: Build Version: 2.17.57_99_gb2219ab [ 2097.268821] LNet: Added LNI 192.168.204.148@tcp [8/256/0/180] [ 2098.993372] Key type lgssc registered [ 2100.101798] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2139.612040] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2139.638050] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2140.885858] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2140.923890] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2141.010604] Lustre: lustre-MDT0000: new disk, initializing [ 2141.104950] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2141.127699] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2145.745045] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 2156.692841] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2156.801631] Lustre: 68533:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2156.853765] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2156.858689] Lustre: Skipped 1 previous similar message [ 2156.930686] Lustre: lustre-MDT0001: new disk, initializing [ 2157.037747] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2157.113782] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2157.127938] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2161.662753] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 2172.114382] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2180.202735] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2189.472540] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2189.804347] Lustre: lustre-OST0000: new disk, initializing [ 2189.809960] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2189.817117] Lustre: 70503:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2189.938353] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2196.566776] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2196.582829] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2196.694948] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2197.029375] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 2206.962759] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2213.074518] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2213.354982] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2218.412214] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2218.632170] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2222.567566] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2222.569846] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2222.571430] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2222.599023] Lustre: Skipped 1 previous similar message [ 2222.630562] LustreError: Skipped 1 previous similar message [ 2226.866347] Lustre: server umount lustre-OST0000 complete [ 2232.805610] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2232.816256] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2232.822890] Lustre: Skipped 1 previous similar message [ 2232.841479] Lustre: Skipped 2 previous similar messages [ 2239.190069] Lustre: server umount lustre-MDT0000 complete [ 2242.980996] LustreError: 68526:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787870694 with bad export cookie 7875171217682337461 [ 2242.983184] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2242.988786] LustreError: 68526:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2243.297594] Lustre: server umount lustre-MDT0001 complete [ 2252.504379] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 18:45:02 (1787870702) [ 2263.763680] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 2308.269587] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_hostid [ 2322.140611] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 2361.910873] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2362.201697] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2362.243935] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2362.369711] Lustre: lustre-MDT0000: new disk, initializing [ 2362.478900] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2362.509837] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2368.329956] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 2379.680151] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2379.788315] Lustre: 76538:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2379.844183] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2379.847865] Lustre: Skipped 1 previous similar message [ 2379.923284] Lustre: lustre-MDT0001: new disk, initializing [ 2380.030165] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2380.065063] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2380.078711] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2384.820623] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 2395.125776] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2400.957526] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2410.149788] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2410.603854] Lustre: lustre-OST0000: new disk, initializing [ 2410.608143] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2410.616538] Lustre: 78507:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2410.787709] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2418.814048] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2418.840779] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2419.021980] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2419.031370] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 2433.340739] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2440.987861] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2441.392940] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2449.128243] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2449.390229] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2455.010778] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2455.013454] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2455.022220] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2455.052192] Lustre: Skipped 1 previous similar message [ 2455.079749] Lustre: Skipped 1 previous similar message [ 2460.138810] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2460.147261] Lustre: Skipped 1 previous similar message [ 2465.252577] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2466.271174] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2466.476934] Lustre: server umount lustre-OST0000 complete [ 2472.416271] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2472.431185] LustreError: Skipped 1 previous similar message [ 2472.439350] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2475.490261] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2475.504765] Lustre: Skipped 3 previous similar messages [ 2478.032670] Lustre: server umount lustre-MDT0000 complete [ 2480.611430] LustreError: 76546:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2480.629725] LustreError: 76546:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 4 previous similar messages [ 2482.767827] LustreError: 76530:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787870934 with bad export cookie 7875171217682338343 [ 2482.771757] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2482.778785] LustreError: 76530:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2483.410240] Lustre: server umount lustre-MDT0001 complete [ 2495.545309] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 18:49:05 (1787870945) [ 2503.223027] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2503.713627] LustreError: 80922:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2503.821551] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2508.877923] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 2509.290397] LustreError: 80923:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2513.391466] LustreError: 80922:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2517.973529] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2523.081866] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 2530.910309] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2538.451331] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2547.425969] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2547.853969] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2547.860100] Lustre: Skipped 1 previous similar message [ 2555.432809] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 2565.272284] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2569.629250] Lustre: 83594:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting 736b4b1e-d1cc-4e88-a43f-4801707df9ae at adminstrative request [ 2573.820724] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2573.840602] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2584.039049] Lustre: MGS: Client 736b4b1e-d1cc-4e88-a43f-4801707df9ae (at 0@lo) reconnecting [ 2584.057315] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x6d4a3a7f7f561599 to 0x6d4a3a7f7f561839 [ 2584.071885] Lustre: MGC192.168.204.148@tcp: Connection restored to 0@lo (at 0@lo) [ 2609.521198] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2609.749417] Lustre: lustre-OST0001: new disk, initializing [ 2609.752746] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2609.758725] Lustre: 83989:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2615.907535] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2615.920344] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2616.057950] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2616.920889] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 2627.421989] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2631.654506] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2631.661450] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2631.663636] LustreError: Skipped 1 previous similar message [ 2631.670531] Lustre: Skipped 1 previous similar message [ 2631.672215] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2631.679433] Lustre: Skipped 1 previous similar message [ 2637.301880] Lustre: server umount lustre-OST0001 complete [ 2641.888265] LustreError: 82745:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2641.913887] LustreError: 82745:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 2645.495136] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2645.504669] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2645.521661] Lustre: Skipped 1 previous similar message [ 2648.604812] Lustre: server umount lustre-OST0000 complete [ 2656.736224] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2656.749625] LustreError: Skipped 1 previous similar message [ 2659.655491] Lustre: server umount lustre-MDT0000 complete [ 2662.370385] LustreError: 80923:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2662.401232] LustreError: 80923:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [ 2663.883461] LustreError: 80904:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787871115 with bad export cookie 7875171217682339897 [ 2663.891780] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2663.900326] LustreError: 80904:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2664.261589] Lustre: server umount lustre-MDT0001 complete [ 2674.630409] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 18:52:04 (1787871124) [ 2676.475569] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2678.667786] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 18:52:08 (1787871128) [ 2680.153674] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2682.219939] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 18:52:12 (1787871132) [ 2683.861389] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2685.842772] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 18:52:16 (1787871136) [ 2687.591987] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2689.576873] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 18:52:19 (1787871139) [ 2691.359303] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2697.660723] loop: module loaded [ 2697.683834] loop0: detected capacity change from 0 to 4294967296 [ 2697.723225] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2699.512214] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2703.561623] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2707.796605] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2736.919292] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2736.976442] LustreError: 88112:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2737.002241] LustreError: 88112:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2737.010923] LustreError: 88112:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2737.017735] LustreError: 88112:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2758.838663] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_hostid [ 2771.561406] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 2817.835389] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2818.208455] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2818.241575] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2818.291412] Lustre: lustre-MDT0000: new disk, initializing [ 2818.354690] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2818.359658] Lustre: Skipped 1 previous similar message [ 2818.370299] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2823.045313] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 2834.165337] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2834.335588] Lustre: 90978:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2834.596167] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2839.595740] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 2849.498305] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2856.080042] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2862.947917] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2863.327423] Lustre: 92947:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2870.384280] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2870.617903] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2871.092889] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 2880.934549] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2886.140758] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2886.438650] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2891.905852] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2892.232444] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2896.360938] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2896.368565] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2896.377176] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2896.378326] Lustre: Skipped 4 previous similar messages [ 2896.394117] Lustre: Skipped 9 previous similar messages [ 2900.523901] Lustre: server umount lustre-OST0000 complete [ 2911.714905] LustreError: 90991:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2911.742149] LustreError: 90991:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 2912.590847] Lustre: server umount lustre-MDT0000 complete [ 2916.155210] LustreError: 94286:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787871367 with bad export cookie 7875171217682340506 [ 2916.159590] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2916.164779] LustreError: 94286:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2916.508576] Lustre: server umount lustre-MDT0001 complete [ 2924.564425] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 18:56:14 (1787871374) [ 2931.226805] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2936.324184] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 2945.587546] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2950.726499] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 2959.249202] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2965.255402] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2971.944918] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2972.370934] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2972.376165] Lustre: Skipped 4 previous similar messages [ 2978.953662] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 2985.743368] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2990.472905] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2990.684956] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2995.267936] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2995.455640] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3002.922438] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3003.094094] Lustre: lustre-OST0001: new disk, initializing [ 3003.097454] Lustre: Skipped 2 previous similar messages [ 3003.103726] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3003.110854] Lustre: Skipped 3 previous similar messages [ 3003.114538] Lustre: 98736:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3010.520704] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 3011.229096] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3011.247221] Lustre: Skipped 2 previous similar messages [ 3011.251683] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3011.350590] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3022.207909] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3027.312415] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 3027.576716] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3032.484397] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 3032.686354] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3053.556059] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 3079.819157] Lustre: Failing over lustre-MDT0000 [ 3080.213480] Lustre: server umount lustre-MDT0000 complete [ 3081.839441] LustreError: 95358:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.204.48@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3081.848893] LustreError: 95358:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 4 previous similar messages [ 3083.236492] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3083.259960] Lustre: Skipped 5 previous similar messages [ 3098.596477] Lustre: 66758:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787871534/real 1787871534] req@00000000da303006 x1874718148024064/t0(0) o400->MGC192.168.204.148@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787871550 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3098.629658] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3105.944124] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 3105.947705] LDISKFS-fs (dm-0): recovery complete [ 3105.955795] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3108.836531] LustreError: 66755:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000724a4015 x1874718148031232/t0(0) o250->MGC192.168.204.148@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3109.590476] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 3112.517618] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 3112.518768] Lustre: 95358:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3114.730717] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 3114.986807] Lustre: 95363:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3114.996349] Lustre: 95363:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 1 previous similar message [ 3115.023838] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3115.097028] LustreError: 101581:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3115.424143] LustreError: 101581:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3115.775236] LustreError: 101581:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3115.781761] LustreError: 101581:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 1 previous similar message [ 3116.103383] LustreError: 101581:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3116.113296] LustreError: 101581:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 1 previous similar message [ 3116.131603] Lustre: 101581:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3116.153372] Lustre: 101581:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 7 previous similar messages [ 3116.824390] LustreError: 101581:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3116.840983] LustreError: 101581:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 2 previous similar messages [ 3117.196943] LustreError: 101581:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3117.210064] LustreError: 101581:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 2 previous similar messages [ 3118.170649] Lustre: 101581:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3118.191491] Lustre: 101581:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 5 previous similar messages [ 3118.841337] LustreError: 101581:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3118.860080] LustreError: 101581:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 5 previous similar messages [ 3119.520673] LustreError: 101581:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3119.530485] LustreError: 101581:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 6 previous similar messages [ 3122.440693] Lustre: 101581:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3122.457942] Lustre: 101581:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 12 previous similar messages [ 3123.095818] LustreError: 101581:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3123.109030] LustreError: 101581:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 12 previous similar messages [ 3123.759130] LustreError: 101581:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3123.776142] LustreError: 101581:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 12 previous similar messages [ 3130.551996] Lustre: 101581:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3130.575383] Lustre: 101581:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 24 previous similar messages [ 3131.234354] LustreError: 101581:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3131.254894] LustreError: 101581:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 24 previous similar messages [ 3131.896327] LustreError: 101581:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3131.909152] LustreError: 101581:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 24 previous similar messages [ 3146.833538] Lustre: 101581:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3146.853900] Lustre: 101581:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 50 previous similar messages [ 3147.507958] LustreError: 101581:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3147.521024] LustreError: 101581:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 50 previous similar messages [ 3148.159235] LustreError: 101581:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3148.173139] LustreError: 101581:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 50 previous similar messages [ 3172.088454] Lustre: 101581:0:(ldlm_lib.c:2457:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3172.100477] Lustre: 101581:0:(ldlm_lib.c:2467:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3172.111576] Lustre: 101581:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client d2ca610b-b5c4-4878-8f35-c47fdb855feb@192.168.204.48@tcp [ 3172.130764] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3172.141167] Lustre: 101581:0:(ldlm_lib.c:1977:abort_req_replay_queue()) @@@ aborted: req@00000000962350b0 x1874718126837760/t0(8589934774) o35->d2ca610b-b5c4-4878-8f35-c47fdb855feb@192.168.204.48@tcp:679/0 lens 392/0 e 0 to 0 dl 1787871634 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3172.224726] Lustre: lustre-MDT0000: Denying connection for new client d2ca610b-b5c4-4878-8f35-c47fdb855feb (at 192.168.204.48@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3172.232785] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3172.266697] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3172.317545] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3172.335841] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3172.337430] Lustre: Skipped 2 previous similar messages [ 3172.377916] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 3172.398239] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:78 to 0x280000401:577) [ 3185.637550] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3185.653123] LustreError: Skipped 2 previous similar messages [ 3185.660026] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3185.686311] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3185.691148] Lustre: Skipped 2 previous similar messages [ 3191.608386] Lustre: server umount lustre-OST0000 complete [ 3191.805541] LustreError: 97181:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3191.848393] LustreError: 97181:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 35 previous similar messages [ 3202.499220] Lustre: server umount lustre-OST0001 complete [ 3213.697570] Lustre: server umount lustre-MDT0000 complete [ 3218.531503] LustreError: 95345:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787871670 with bad export cookie 7875171217682391732 [ 3218.534281] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3218.555693] LustreError: 95345:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3218.918917] Lustre: server umount lustre-MDT0001 complete [ 3227.753457] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 19:01:18 (1787871678) [ 3235.820187] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3235.824214] Lustre: *** cfs_fail_loc=197, val=0*** [ 3251.359640] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3256.479494] LustreError: 103569:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.148@tcp: failed processing log, type 4: rc = -110 [ 3261.636936] Lustre: *** cfs_fail_loc=197, val=0*** [ 3261.644262] Lustre: Skipped 65 previous similar messages [ 3282.207457] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3282.215634] Lustre: Skipped 2 previous similar messages [ 3290.823150] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 3300.359184] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3302.926989] Lustre: server umount lustre-OST0000 complete [ 3313.124634] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 19:02:43 (1787871763) [ 3336.556223] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3337.194170] LustreError: 105189:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3337.235367] LustreError: 105189:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 6 previous similar messages [ 3343.194143] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 3355.128474] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3359.478536] LustreError: 105220:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3359.490439] LustreError: 105220:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3359.501606] LustreError: 105220:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 22, retries 0, failed: rc = -5 [ 3359.907475] Lustre: server umount lustre-MDT0000 complete [ 3394.165559] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_hostid [ 3407.417773] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 3449.009852] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3449.342549] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3449.383970] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3449.546915] Lustre: lustre-MDT0000: new disk, initializing [ 3449.725778] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3456.241453] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 3469.341988] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3469.480380] Lustre: 109399:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3469.920846] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3475.068921] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 3483.920549] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3489.396338] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3497.164587] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3497.499619] Lustre: 111370:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3503.553785] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 3504.907501] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3515.080166] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3521.728915] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3522.063747] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3528.660660] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3529.051702] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3535.841917] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3535.850645] LustreError: Skipped 1 previous similar message [ 3535.854147] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3535.856977] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3535.864768] Lustre: Skipped 6 previous similar messages [ 3535.869203] Lustre: Skipped 9 previous similar messages [ 3537.499599] Lustre: server umount lustre-OST0000 complete [ 3547.712239] Lustre: server umount lustre-MDT0000 complete [ 3551.722968] LustreError: 110204:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787872003 with bad export cookie 7875171217682402365 [ 3551.726215] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3551.735275] LustreError: 110204:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3552.141432] Lustre: server umount lustre-MDT0001 complete [ 3562.120748] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 19:06:52 (1787872012) [ 3572.330727] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing unload_modules_local [ 3575.765590] Key type lgssc unregistered [ 3576.187561] LNet: 113860:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3576.206935] LNetError: 113860:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3576.239530] LNet: Removed LNI 192.168.204.148@tcp [ 3577.076923] Key type .llcrypt unregistered [ 3577.082580] Key type ._llcrypt unregistered [ 3596.965339] Key type ._llcrypt registered [ 3596.968297] Key type .llcrypt registered [ 3597.062851] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_hostid [ 3615.369053] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 3617.262727] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3617.334883] alg: No test for adler32 (adler32-zlib) [ 3618.501713] Lustre: Lustre: Build Version: 2.17.57_99_gb2219ab [ 3618.870028] LNet: Added LNI 192.168.204.148@tcp [8/256/0/180] [ 3620.599256] Key type lgssc registered [ 3622.007902] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3673.316858] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3673.352772] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3674.615509] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3674.667822] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3674.769902] Lustre: lustre-MDT0000: new disk, initializing [ 3674.869546] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3674.919339] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3680.094683] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 3692.007841] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3692.321895] Lustre: lustre-OST0000: new disk, initializing [ 3692.325762] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3692.330519] Lustre: Skipped 1 previous similar message [ 3692.334438] Lustre: 118036:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3692.405887] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3698.821990] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 3699.375359] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3699.381622] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3699.416570] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3710.877351] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3711.044590] Lustre: lustre-OST0001: new disk, initializing [ 3711.055365] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3711.065601] Lustre: 118901:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3711.201592] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3718.168040] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 3719.392663] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3719.411579] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3719.461239] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3729.894741] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3729.911273] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3729.918399] Lustre: Skipped 1 previous similar message [ 3733.756221] Lustre: server umount lustre-MDT0000 complete [ 3741.454706] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3746.746549] LustreError: 117221:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787872198 with bad export cookie 2944952849062167674 [ 3746.754266] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3746.761172] LustreError: 117221:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3746.947468] Lustre: server umount lustre-OST0000 complete [ 3751.553406] Lustre: server umount lustre-OST0001 complete [ 3762.493229] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 19:10:12 (1787872212) [ 3791.518464] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 19:10:41 (1787872241) [ 3803.724948] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 3827.448867] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 19:11:17 (1787872277) [ 3867.090492] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_hostid [ 3877.871970] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 3915.218680] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3915.557632] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3915.607967] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3915.696790] Lustre: lustre-MDT0000: new disk, initializing [ 3915.766144] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3915.790253] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3920.351890] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 3930.913377] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3931.027412] Lustre: 126656:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3931.093411] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3931.099506] Lustre: Skipped 1 previous similar message [ 3931.208899] Lustre: lustre-MDT0001: new disk, initializing [ 3931.310314] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3931.345487] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3931.364846] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3935.903853] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 3946.215630] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3952.682682] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3959.557300] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3959.927325] Lustre: lustre-OST0000: new disk, initializing [ 3959.932462] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3959.942765] Lustre: 128625:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3960.055563] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3966.036779] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3966.054237] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3966.211321] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3966.753186] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 3976.796669] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3981.462793] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3981.656530] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3986.161942] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3986.424139] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3992.032362] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3992.038645] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3992.049708] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3992.060943] Lustre: Skipped 2 previous similar messages [ 3992.073381] Lustre: Skipped 1 previous similar message [ 3994.712673] Lustre: server umount lustre-OST0000 complete [ 4002.277111] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4002.281720] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4002.287857] Lustre: Skipped 1 previous similar message [ 4002.299796] Lustre: Skipped 1 previous similar message [ 4005.537992] Lustre: server umount lustre-MDT0000 complete [ 4009.226449] LustreError: 126649:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787872460 with bad export cookie 2944952849062168899 [ 4009.235246] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4009.245725] LustreError: 126649:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 4009.812980] Lustre: server umount lustre-MDT0001 complete [ 4021.657715] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 19:14:31 (1787872471) [ 4028.596847] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4029.039574] LustreError: 131036:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4029.057318] LustreError: 131036:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [ 4029.109117] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4033.703136] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 4034.539401] LustreError: 131037:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4038.626578] LustreError: 131036:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4041.960000] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4046.490301] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 4054.173558] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4059.683828] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4065.166388] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4065.518244] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4065.524362] Lustre: Skipped 1 previous similar message [ 4071.353906] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 4080.233796] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4085.280663] Lustre: *** cfs_fail_loc=159, val=0*** [ 4101.806419] Lustre: lustre-MDT0000: Client 23d8956c-8a93-4e4f-8c21-4989cd7685b3 (at 192.168.204.48@tcp) reconnecting [ 4103.455597] Lustre: *** cfs_fail_loc=159, val=0*** [ 4103.466140] Lustre: Skipped 5 previous similar messages [ 4119.182661] Lustre: lustre-MDT0000: Client 23d8956c-8a93-4e4f-8c21-4989cd7685b3 (at 192.168.204.48@tcp) reconnecting [ 4138.464361] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4138.486153] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4138.524333] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4140.512680] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4143.267231] Lustre: server umount lustre-OST0000 complete [ 4149.728562] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4149.735987] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4149.749877] Lustre: Skipped 1 previous similar message [ 4149.761062] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4149.773611] Lustre: Skipped 2 previous similar messages [ 4153.352332] Lustre: server umount lustre-MDT0000 complete [ 4155.882291] LustreError: 133636:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4155.906903] LustreError: 133636:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 4156.886215] LustreError: 131016:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787872608 with bad export cookie 2944952849062169781 [ 4156.887714] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4156.894133] LustreError: 131016:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4157.156474] Lustre: server umount lustre-MDT0001 complete [ 4168.358216] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing unload_modules_local [ 4170.963612] Key type lgssc unregistered [ 4171.243372] LNet: 134830:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4171.254777] LNetError: 134830:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4171.280957] LNet: Removed LNI 192.168.204.148@tcp [ 4172.217856] Key type .llcrypt unregistered [ 4172.223251] Key type ._llcrypt unregistered [ 4195.077054] Key type ._llcrypt registered [ 4195.080691] Key type .llcrypt registered [ 4195.208758] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 4196.358991] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4196.435846] alg: No test for adler32 (adler32-zlib) [ 4197.455847] Lustre: Lustre: Build Version: 2.17.57_99_gb2219ab [ 4197.673562] LNet: Added LNI 192.168.204.148@tcp [8/256/0/180] [ 4199.319219] Key type lgssc registered [ 4200.318631] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4208.045441] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4208.065176] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4209.443988] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4213.935453] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 4219.877139] LustreError: 136244:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4219.898220] LustreError: 136244:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 4222.853918] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4223.193096] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4227.414293] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 4235.220586] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4240.719873] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4247.893505] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4248.039836] LustreError: 138061:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4248.278634] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4254.679764] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 4254.773487] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4262.911857] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4272.015478] Lustre: *** cfs_fail_loc=159, val=0*** [ 4287.631115] Lustre: lustre-MDT0000: Client c78e18ca-30b8-4ebe-a790-bd7b398323e9 (at 192.168.204.48@tcp) reconnecting [ 4290.267227] Lustre: *** cfs_fail_loc=159, val=0*** [ 4306.579224] Lustre: lustre-MDT0001: Client c78e18ca-30b8-4ebe-a790-bd7b398323e9 (at 192.168.204.48@tcp) reconnecting [ 4308.331165] Lustre: *** cfs_fail_loc=159, val=0*** [ 4308.336262] Lustre: Skipped 3 previous similar messages [ 4325.020323] Lustre: lustre-MDT0001: Client c78e18ca-30b8-4ebe-a790-bd7b398323e9 (at 192.168.204.48@tcp) reconnecting [ 4340.650175] Lustre: 139301:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4345.012843] Lustre: *** cfs_fail_loc=159, val=0*** [ 4345.019193] Lustre: Skipped 5 previous similar messages [ 4360.366295] Lustre: lustre-MDT0000: Client b6c64621-515f-44a9-82ac-94d30ff595ca (at 192.168.204.48@tcp) reconnecting [ 4362.523654] Lustre: *** cfs_fail_loc=159, val=0*** [ 4362.531740] Lustre: Skipped 14 previous similar messages [ 4378.265335] Lustre: lustre-MDT0000: Client b6c64621-515f-44a9-82ac-94d30ff595ca (at 192.168.204.48@tcp) reconnecting [ 4380.191242] Lustre: 139595:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4385.768196] LustreError: lustre-OST0000-osc-MDT0001: operation ost_destroy to node 0@lo failed: rc = -107 [ 4385.769775] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4385.783336] LustreError: Skipped 2 previous similar messages [ 4385.801143] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4387.809468] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4387.813665] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4387.817423] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4387.822867] LustreError: Skipped 2 previous similar messages [ 4392.021194] Lustre: server umount lustre-OST0000 complete [ 4397.544181] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4397.553567] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4397.578050] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4397.585441] Lustre: Skipped 1 previous similar message [ 4399.827780] Lustre: server umount lustre-MDT0000 complete [ 4403.169330] LustreError: 139362:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4403.195427] LustreError: 139362:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 4405.478716] LustreError: 139179:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787872857 with bad export cookie 6535353829884126760 [ 4405.483947] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4405.490728] LustreError: 139179:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4405.858943] Lustre: server umount lustre-MDT0001 complete [ 4420.192477] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing unload_modules_local [ 4423.533185] Key type lgssc unregistered [ 4423.929973] LNet: 140651:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4423.939847] LNetError: 140651:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4425.002382] LNet: Removed LNI 192.168.204.148@tcp [ 4425.935440] Key type .llcrypt unregistered [ 4425.937579] Key type ._llcrypt unregistered [ 4457.605127] Key type ._llcrypt registered [ 4457.608189] Key type .llcrypt registered [ 4457.725311] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 4458.900944] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4459.003163] alg: No test for adler32 (adler32-zlib) [ 4460.121766] Lustre: Lustre: Build Version: 2.17.57_99_gb2219ab [ 4460.313109] LNet: Added LNI 192.168.204.148@tcp [8/256/0/180] [ 4461.991176] Key type lgssc registered [ 4463.102749] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4472.052142] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4472.073156] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4473.671769] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4477.763828] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 4484.069194] LustreError: 142069:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4484.095697] LustreError: 142069:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 4488.110143] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4488.518453] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4493.899333] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 4503.581701] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4511.755829] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4520.268614] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4520.714262] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4527.499602] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 4535.126457] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4536.612091] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4544.306093] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4553.676332] Lustre: 144677:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4556.762471] Lustre: 144725:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4558.748450] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4559.632859] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4564.419170] Lustre: 144871:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4565.465506] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4566.353509] Lustre: 144970:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4575.210083] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4575.222788] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4575.225384] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4575.244234] Lustre: Skipped 1 previous similar message [ 4575.265219] Lustre: Skipped 1 previous similar message [ 4577.493541] Lustre: server umount lustre-OST0000 complete [ 4585.441643] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4585.450193] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4585.456833] Lustre: Skipped 1 previous similar message [ 4590.564035] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4590.581684] Lustre: Skipped 2 previous similar messages [ 4595.682025] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4595.695234] Lustre: Skipped 1 previous similar message [ 4596.191144] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4596.441036] Lustre: server umount lustre-MDT0000 complete [ 4600.802974] LustreError: 142069:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4600.818845] LustreError: 142069:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 4601.035397] LustreError: 142050:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787873052 with bad export cookie 16842525391797520416 [ 4601.049087] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4601.374628] Lustre: server umount lustre-MDT0001 complete [ 4614.248981] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing unload_modules_local [ 4617.721451] Key type lgssc unregistered [ 4618.146964] LNet: 146035:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4618.154969] LNetError: 146035:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4618.173944] LNet: Removed LNI 192.168.204.148@tcp [ 4619.140994] Key type .llcrypt unregistered [ 4619.143841] Key type ._llcrypt unregistered [ 4647.886464] Key type ._llcrypt registered [ 4647.888498] Key type .llcrypt registered [ 4648.032446] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 4648.956969] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4648.990524] alg: No test for adler32 (adler32-zlib) [ 4650.229537] Lustre: Lustre: Build Version: 2.17.57_99_gb2219ab [ 4650.550333] LNet: Added LNI 192.168.204.148@tcp [8/256/0/180] [ 4652.264257] Key type lgssc registered [ 4653.350098] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4661.315333] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4661.332206] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4662.830824] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4667.974556] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 4673.001854] LustreError: 147452:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4673.021235] LustreError: 147452:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 4678.112783] LustreError: 147471:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4679.430546] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4680.072376] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4686.346365] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 4696.533262] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4705.293093] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4715.000200] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4715.446304] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4723.474728] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 4724.670133] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4732.717629] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4732.868959] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4739.964852] Lustre: *** cfs_fail_loc=159, val=0*** [ 4755.645914] Lustre: lustre-MDT0000: Client ace7e4f2-9c5b-435f-892b-72fa571d4ca2 (at 192.168.204.48@tcp) reconnecting [ 4763.625509] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4763.628015] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4763.647832] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4763.674883] Lustre: Skipped 1 previous similar message [ 4768.218932] Lustre: server umount lustre-OST0000 complete [ 4777.446459] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4777.462863] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4777.474026] Lustre: Skipped 1 previous similar message [ 4777.487667] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4778.977172] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4778.980554] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4780.537301] Lustre: server umount lustre-MDT0000 complete [ 4784.109618] LustreError: 147453:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4784.139867] LustreError: 147453:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 4785.389811] LustreError: 147433:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787873237 with bad export cookie 3546586746878140418 [ 4785.418950] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4785.913349] Lustre: server umount lustre-MDT0001 complete [ 4800.495588] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing unload_modules_local [ 4803.989369] Key type lgssc unregistered [ 4804.331813] LNet: 151120:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4804.338878] LNetError: 151120:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4804.361164] LNet: Removed LNI 192.168.204.148@tcp [ 4805.621422] Key type .llcrypt unregistered [ 4805.624403] Key type ._llcrypt unregistered [ 4837.100358] Key type ._llcrypt registered [ 4837.102852] Key type .llcrypt registered [ 4837.244489] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 4838.619221] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4838.650839] alg: No test for adler32 (adler32-zlib) [ 4839.752112] Lustre: Lustre: Build Version: 2.17.57_99_gb2219ab [ 4839.981401] LNet: Added LNI 192.168.204.148@tcp [8/256/0/180] [ 4841.687243] Key type lgssc registered [ 4843.316756] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4853.642487] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4853.658796] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4855.235055] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4861.662276] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 4864.497050] LustreError: 152536:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4864.512899] LustreError: 152536:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 4869.611967] LustreError: 152535:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4872.206755] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4872.686532] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4877.758630] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 4886.365777] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4893.781553] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4902.182751] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4902.640024] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4910.123564] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 4919.037259] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4920.595521] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4925.233623] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4945.422125] Lustre: lustre-OST0000: evicting 68169598-2696-4f07-b0ec-147a5a7d18b0 (at 192.168.204.48@tcp) by administrative request [ 4945.428901] Lustre: Skipped 1 previous similar message [ 4975.584791] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4975.597627] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4975.611286] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4976.617905] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4976.626981] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4976.627151] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4976.629468] Lustre: Skipped 1 previous similar message [ 4981.748759] Lustre: server umount lustre-OST0000 complete [ 4986.173709] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4986.175548] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4986.197045] Lustre: Skipped 1 previous similar message [ 4991.456726] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4991.471503] Lustre: Skipped 2 previous similar messages [ 4992.579743] Lustre: server umount lustre-MDT0000 complete [ 4996.578302] LustreError: 153266:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4996.608365] LustreError: 153266:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 4997.419268] LustreError: 152517:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787873449 with bad export cookie 2555635310316979411 [ 4997.463900] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4998.093936] Lustre: server umount lustre-MDT0001 complete [ 5012.463178] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing unload_modules_local [ 5015.333747] Key type lgssc unregistered [ 5015.648648] LNet: 156608:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5015.656092] LNetError: 156608:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5015.677457] LNet: Removed LNI 192.168.204.148@tcp [ 5016.834521] Key type .llcrypt unregistered [ 5016.836364] Key type ._llcrypt unregistered [ 5109.921255] Key type ._llcrypt registered [ 5109.923526] Key type .llcrypt registered [ 5110.024669] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 5111.648704] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5111.728859] alg: No test for adler32 (adler32-zlib) [ 5112.846354] Lustre: Lustre: Build Version: 2.17.57_99_gb2219ab [ 5113.127123] LNet: Added LNI 192.168.204.148@tcp [8/256/0/180] [ 5114.895197] Key type lgssc registered [ 5116.189091] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5124.567123] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5124.589465] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5126.065202] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5131.019365] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 5135.335195] LustreError: 159103:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5135.388135] LustreError: 159103:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 5140.455676] LustreError: 159104:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5141.238965] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5141.706552] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5147.732528] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 5157.399423] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5164.068220] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5173.345497] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5173.939068] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5181.226251] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 5187.261724] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5191.046825] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5195.458437] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 5205.987527] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5205.988588] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5205.998875] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5211.108811] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5211.119581] Lustre: Skipped 2 previous similar messages [ 5212.222306] Lustre: server umount lustre-OST0000 complete [ 5218.275426] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5218.286954] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5218.300618] Lustre: Skipped 1 previous similar message [ 5218.309584] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5221.347215] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5221.350581] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5223.877943] Lustre: server umount lustre-MDT0000 complete [ 5226.465967] LustreError: 159103:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5226.501439] LustreError: 159103:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 5228.837859] LustreError: 159085:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787873680 with bad export cookie 5715418044516103658 [ 5228.842652] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5228.858192] LustreError: 159085:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5229.486353] Lustre: server umount lustre-MDT0001 complete [ 5241.948442] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing unload_modules_local [ 5245.017993] Key type lgssc unregistered [ 5245.371301] LNet: 162679:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5245.383787] LNetError: 162679:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5245.402096] LNet: Removed LNI 192.168.204.148@tcp [ 5246.488812] Key type .llcrypt unregistered [ 5246.500323] Key type ._llcrypt unregistered [ 5274.997807] Key type ._llcrypt registered [ 5274.999640] Key type .llcrypt registered [ 5275.140815] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 5276.686865] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5276.824203] alg: No test for adler32 (adler32-zlib) [ 5278.565458] Lustre: Lustre: Build Version: 2.17.57_99_gb2219ab [ 5279.082587] LNet: Added LNI 192.168.204.148@tcp [8/256/0/180] [ 5280.857414] Key type lgssc registered [ 5282.485906] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5301.217531] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 19:35:51 (1787873751) [ 5319.774956] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_hostid [ 5330.146389] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 5369.970810] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5370.000891] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5371.329808] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5371.371655] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5371.466422] Lustre: lustre-MDT0000: new disk, initializing [ 5371.573619] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5371.598840] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5376.527641] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 5388.589870] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5388.760692] Lustre: 166949:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5388.857875] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5388.864628] Lustre: Skipped 1 previous similar message [ 5388.953550] Lustre: lustre-MDT0001: new disk, initializing [ 5389.108753] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5389.190236] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5389.203783] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5394.848449] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 5405.446403] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5413.051954] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5420.750676] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5421.014985] Lustre: lustre-OST0000: new disk, initializing [ 5421.019349] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5421.029366] Lustre: 168915:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5421.119228] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5427.725878] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 5429.312119] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5429.320997] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5429.414197] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5437.669102] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5450.217939] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5450.220595] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5450.233646] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5450.248153] Lustre: Skipped 1 previous similar message [ 5454.937963] Lustre: server umount lustre-OST0000 complete [ 5460.448568] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5460.461189] Lustre: Skipped 1 previous similar message [ 5460.472576] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5460.479598] Lustre: Skipped 1 previous similar message [ 5465.564616] Lustre: server umount lustre-MDT0000 complete [ 5470.559534] LustreError: 166942:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787873922 with bad export cookie 11997459444176786369 [ 5470.571375] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5470.575680] LustreError: 166942:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5471.155526] Lustre: server umount lustre-MDT0001 complete [ 5483.434647] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing unload_modules_local [ 5486.578195] Key type lgssc unregistered [ 5486.907169] LNet: 170824:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5486.923035] LNetError: 170824:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5486.941941] LNet: Removed LNI 192.168.204.148@tcp [ 5487.880862] Key type .llcrypt unregistered [ 5487.884524] Key type ._llcrypt unregistered [ 5513.577367] Key type ._llcrypt registered [ 5513.579956] Key type .llcrypt registered [ 5513.678817] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 5515.089340] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5515.150026] alg: No test for adler32 (adler32-zlib) [ 5516.147371] Lustre: Lustre: Build Version: 2.17.57_99_gb2219ab [ 5516.394996] LNet: Added LNI 192.168.204.148@tcp [8/256/0/180] [ 5518.063153] Key type lgssc registered [ 5519.196816] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5526.841579] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5526.858635] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5528.183311] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5532.618214] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 5537.776165] LustreError: 172241:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5537.793209] LustreError: 172241:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 5540.987642] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5541.367855] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5546.303049] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 5553.959459] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5559.028964] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5566.255437] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5566.616818] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5571.674954] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 5572.785968] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5579.264925] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5587.327508] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5587.354798] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5587.370637] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5587.377091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.392729] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5588.647139] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5588.696367] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5588.721931] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5588.738158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.762662] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5590.078031] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5590.105163] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5590.124156] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5590.133122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.151498] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5591.252295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.223621] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5592.232573] Lustre: Skipped 1 previous similar message [ 5592.258629] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5592.273500] Lustre: Skipped 1 previous similar message [ 5592.281150] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5592.287289] Lustre: Skipped 1 previous similar message [ 5592.291528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.305933] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5592.319184] Lustre: Skipped 1 previous similar message [ 5593.159776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.200620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.478773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.534988] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5596.545921] Lustre: Skipped 3 previous similar messages [ 5596.571684] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5596.580435] Lustre: Skipped 3 previous similar messages [ 5596.584827] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5596.588888] Lustre: Skipped 3 previous similar messages [ 5596.594947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.601822] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5596.612153] Lustre: Skipped 3 previous similar messages [ 5597.727619] LustreError: 172261:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5597.739404] LustreError: 172261:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5597.768784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.790443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.594684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.518783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.568057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.523294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.608147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.552322] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5604.560169] Lustre: Skipped 7 previous similar messages [ 5604.578709] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5604.590320] Lustre: Skipped 7 previous similar messages [ 5604.596214] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5604.606978] Lustre: Skipped 7 previous similar messages [ 5604.612928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.625185] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5604.634111] Lustre: Skipped 7 previous similar messages [ 5605.854701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.750820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.692394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.625620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.587613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.518194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.513119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.356328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.468703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.783681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.873668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.073801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.005958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.108047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.145191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.365064] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5621.371764] Lustre: Skipped 15 previous similar messages [ 5621.398821] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5621.415345] Lustre: Skipped 15 previous similar messages [ 5621.421934] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5621.429942] Lustre: Skipped 15 previous similar messages [ 5621.438343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.448026] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5621.463636] Lustre: Skipped 15 previous similar messages [ 5622.365570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.344247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.459686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.543916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.525939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.522700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.454507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.498475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.461260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.590200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.390742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.368392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.327366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.527573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.827065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.004079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.370542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.804735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.812440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.895405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.031695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.944295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.940525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.808039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.708707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.514327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.303331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.309080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.258161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.153419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.999751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.672533] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5653.676944] Lustre: Skipped 31 previous similar messages [ 5653.686749] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5653.694392] Lustre: Skipped 31 previous similar messages [ 5653.697885] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5653.703269] Lustre: Skipped 31 previous similar messages [ 5653.709734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.722499] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5653.726542] Lustre: Skipped 31 previous similar messages [ 5654.558785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.383053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.314878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.272302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.202928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.429086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.388290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.578589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.971635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.089114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.944020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.931310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.765301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.686626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.674062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.781517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.823382] LustreError: 172261:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5670.840204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.844056] LustreError: 172261:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5671.710188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.737226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.684230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.665510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.689230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.755838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.733081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.655545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.553803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.556681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.485252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.674161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.676870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.947684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.257732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.430621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.421274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.467788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.515520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.705562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.823103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.973866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.806729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.967490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.237866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.288069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.504506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.547735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.579796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.757356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.947211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.046356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.227733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.982952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.995748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.409778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.574769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.542544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.723464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.058626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.117964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.908364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.740210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.766505] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5717.775374] Lustre: Skipped 60 previous similar messages [ 5717.796981] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5717.815576] Lustre: Skipped 60 previous similar messages [ 5717.820684] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5717.830448] Lustre: Skipped 60 previous similar messages [ 5717.835766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.846737] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5717.853340] Lustre: Skipped 60 previous similar messages [ 5718.772406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.733083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.740091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.598884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.639086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.710175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.808190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.666967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.586900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.438428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.391341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.261337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.258212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.345615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.378829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.363133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.294587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.225291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.101535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.160420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.241191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.420207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.449463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.295105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.310806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.262052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.275647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.144833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.039610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.792316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.852886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.720236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.757629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.878162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.794240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.765301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.690627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.634315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.549627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.366304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.420251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.410277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.351414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.188724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.192320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.139782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.207708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.068363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.242104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.257148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.425724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.369520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.483080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.401935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.755322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.476962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.851631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.973428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.028037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.020558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.953303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.215210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.094863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.201876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.154148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.100259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.008956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.945551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.206235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.469363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.682535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.699383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.708802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.931833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.022675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.936108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.807041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.751193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.065089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.214940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.188777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.124535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.993429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.080225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.885325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.939779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.861833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.842351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.682933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.711533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.471183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.352977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.219606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.191064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.207315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.172712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.018592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.895419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.680884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.645789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.641386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.801912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.869393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.035697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.972675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.041201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.088339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.617336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.737309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.668372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.570732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.603699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.714341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.206058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.629160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.835950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.812919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.951552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.928048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.090173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.096530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.149721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.063944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.056977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.066633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.029030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.052776] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5846.064414] Lustre: Skipped 126 previous similar messages [ 5846.109541] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5846.124539] Lustre: Skipped 126 previous similar messages [ 5846.128868] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5846.138559] Lustre: Skipped 126 previous similar messages [ 5846.145176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.158344] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5846.171670] Lustre: Skipped 126 previous similar messages [ 5847.174539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.364479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.570734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.877116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.943136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.955311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.124795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.112506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.132327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.075734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.313150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.466836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.550732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.617143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.529081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.755789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.830994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.757463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.761962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.731154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.717927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.675729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.821989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.956943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.873395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.932991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.863708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.005384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.022236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.042735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.914592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.954039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.254748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.307927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.365660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.474220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.607030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.526697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.649679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.738065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.950686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.226923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.278871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.232154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.444860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.423153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.939756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.189483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.442210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.724624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.836671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.196716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.453639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.751620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.960571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.815260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.979936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.994891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.375964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.513151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.313303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.551354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.615448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.668731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.946982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.150922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.564485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.915621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.096094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.354435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.424168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.494876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.848497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.807468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.019622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.277415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.310570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.417184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.627446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.988947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.379092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.784290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.959064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.777871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.975856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.874834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.054174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.376531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.561633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.574700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.765445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.905092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.881424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.179771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.999140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.277646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.552759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.552247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.541328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.307627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.350862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.445098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.390945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.336906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.434912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.400927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.385240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.381291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.395655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.406622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.565550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.433081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.494616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.544049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.678244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.839760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.791796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.143542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.433789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.836578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.907462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.063443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.228318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.008457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.028741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.226372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.556183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.772788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.973365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.513147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.514131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.721941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.233748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.558573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.921289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.168189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.222220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.479400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.465271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.507908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.532148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.603680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.924293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.179251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.517638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.852774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.127161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.082539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.176691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.301711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.443296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.418593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.386707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.409092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.421444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.555805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.540241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.647722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.594492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.638141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.577668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.525642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.573214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.524485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.519754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.361731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.380115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.189309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.110932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.894522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.771257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.690723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.490574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.411523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.289817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.251922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.121584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.210595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.075894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.053428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.168821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.982393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.156281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.090468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.101577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.129458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.086722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.097437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.106452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.085023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.250201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.160877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.083570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.094727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.062792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.008358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.077745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.980740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.971536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.890879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.916881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.682157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.603682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.476752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.316943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.249498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.371985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.423068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.452115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.499716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.479689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.399139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.380750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.406894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.510403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.544949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.675734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.786185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.800775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.857052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.127243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.144455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.092525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.978476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.046234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.907174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.984670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.937079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.052150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.259586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.310362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.323818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.175613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.134672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.988507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.163975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.193869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.148829] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6102.157170] Lustre: Skipped 237 previous similar messages [ 6102.186665] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6102.197028] Lustre: Skipped 237 previous similar messages [ 6102.206656] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6102.220534] Lustre: Skipped 237 previous similar messages [ 6102.224162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.244370] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6102.252049] Lustre: Skipped 237 previous similar messages [ 6103.130781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.094446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.073322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.219408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.453818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.835482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.743336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.718944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.001252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.952781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.137778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.305790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.204507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.323687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.211797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.036738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.993406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.797687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.772848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.828959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.774420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.802791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.710571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.768821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.706962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.717681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.605527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.599261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.613961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.683625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.588768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.466304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.745337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.843575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.846428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.785302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.715539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.759752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.714234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.808108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.880322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.928880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.901249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.995147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.123918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.194807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.348075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.684713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.865068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.016770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.256150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.445224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.742101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.019888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.215881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.453600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.905778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.197730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.555048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.860741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.162071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.716647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.840354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.911940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.118333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.457116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.378908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.446180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.413389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.444350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.492404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.743422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.963915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.847500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.106633] LustreError: 172261:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6184.116289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.118993] LustreError: 172261:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6185.227108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.160353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.263915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.276960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.517985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.615966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.667068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.685559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.722987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.650937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.539691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.700924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.892824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.022916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.997135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.084631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.018837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.281487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.473382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.358551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.507906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.583330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.701166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.835543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.965572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.092906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.203244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.310255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.416292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.684619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.472868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.722477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.783459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.780876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.116119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.481042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.639384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.160805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.243065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.273976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.405392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.706035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.225609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.564716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.754730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.725825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.881345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.029389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.007409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.126188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.230931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.325195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.153988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.259606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.323722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.406350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.403220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.745626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.175096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.702489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.920221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.409521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.489804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.889511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.173834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.334285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.457361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.252610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.236918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.474959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.647797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.963256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.191945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.332241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.469819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.429190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.502696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.521902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.887114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.425040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.716240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.937944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.366529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.872707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.127942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.458365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.564759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.900860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.516877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.586429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.470276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.522401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.383622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.336249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.276418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.341061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.269735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.235756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.126586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.121517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.981263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.994443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.820628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.746605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.706793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.904640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.923436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.943962] LustreError: 172261:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6305.959494] LustreError: 172261:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6305.970026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.812836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.002703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.007555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.937236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.144595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.236963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.337832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.340795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.486242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.335986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.308319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.316692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6319.552655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.660507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.577435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.716443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.795802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.999660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.190167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.205365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.486667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.688323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.917213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.015806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.304799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.540943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.957361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.778146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6338.773977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.883246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.153829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6342.507620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6343.718246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.630821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.048313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.126611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6348.387452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.555720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.945751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.216425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.411779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6354.764102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.100788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.181578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.256564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.423738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.742052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6361.736521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6362.741471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.700563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.043624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.333140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.479896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.877504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.135475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.307271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.450406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6373.292068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.400630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.419379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.464657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6377.375941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.357604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.218654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.036574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.903243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.887061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6382.900388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6384.019028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6385.250971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6386.457893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.652549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6388.928219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6390.318038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.550871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6392.863137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6394.101806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6395.418923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6396.705872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.910651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.977151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.109189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6401.341672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.315752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6403.606594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6404.797989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6406.132952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6407.090097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6408.118704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6409.255324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.637491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.803304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6412.898079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6414.107811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.385819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.527622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.856553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.076839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.354974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6421.369189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6422.586319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6423.710354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.540886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.538858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6426.483551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6427.547340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6428.731275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6429.799446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.752890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.681462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6432.579626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6433.510921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6434.529323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.633588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6436.485511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6437.525521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.408204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6439.526569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6440.323892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.250948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.281801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6443.360294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6444.345823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.388499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6446.540767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6447.526533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6448.682960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6449.688612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.093861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6452.329905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6453.165046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6454.156739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6454.981218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.873303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6456.845836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6457.916413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6459.261474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6460.384022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6461.410769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6462.404473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6463.436121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6464.353437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6465.269714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6466.243740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6467.190975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6467.993640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6468.955151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6469.943939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6471.021699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6471.865697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6472.821354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6473.656201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6474.500894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6475.343792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6476.166203] LustreError: 172261:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6476.177232] LustreError: 172261:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6476.184064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6477.065986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6478.039393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6478.998301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6480.103091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6481.172637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6482.115056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6482.900629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6483.823508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6484.568166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6485.556720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6486.427747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6487.355823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6488.257092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6489.121744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6489.993254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6490.744342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6491.514963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6492.344136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6493.177617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6493.983320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6494.844675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6495.746655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6496.500827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6497.259639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6497.963691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6498.829532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6499.666228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6500.529471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6501.451186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6502.551554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6503.524348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6504.367575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6505.134132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6505.955239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6506.785637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6507.712835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6508.508334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6509.371347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6510.298514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6511.073318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6511.890593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6512.818252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6513.758193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6514.634437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6515.515583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6516.338238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6517.237319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6518.218417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6519.446403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6520.293694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6521.236156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6522.062592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6522.806107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6523.776909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6524.613904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6525.738835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6526.750451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6527.908895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6528.959112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6530.163354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6531.145938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6532.489475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6533.390431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6534.346340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6535.092668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6535.902743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6536.773381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6537.721357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6538.520920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6539.303995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6540.356025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6541.428794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6542.485692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6543.347869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6544.162708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6544.930214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6545.770569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6546.701176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6547.725205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6548.473139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6549.359958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6550.390749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6551.193829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6551.968808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6552.740525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6553.570373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6554.501621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6555.336115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6556.266361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6557.089292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6557.990841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6558.938805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6559.915142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6560.851297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6561.707212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6562.836216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6563.829701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6564.630968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6565.304724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6566.097661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6566.961860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6567.862947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6568.664443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6569.474982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6570.555917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6571.664798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6572.740640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6573.803176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6574.733196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6575.670902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6576.792254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6577.866128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6579.050505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6580.050376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6581.327848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6582.299756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6583.330077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6584.406487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6585.285674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6586.137205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6587.119777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6588.079863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6589.098791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6589.892795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6590.694194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6591.507259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6592.532473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6593.322052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6594.403076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6595.201354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6596.063715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6596.961708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6597.897605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6598.792055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6599.546967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6600.499734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6601.411632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6602.276992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6603.205894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6603.998970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6604.888935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6605.804502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6606.767594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6607.872300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6608.863980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6609.784703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6610.744210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6611.651557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6612.648947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6613.478070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6614.274265] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6614.279412] Lustre: Skipped 488 previous similar messages [ 6614.291962] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6614.301030] Lustre: Skipped 488 previous similar messages [ 6614.308581] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6614.318617] Lustre: Skipped 488 previous similar messages [ 6614.324833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6614.333590] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6614.338694] Lustre: Skipped 488 previous similar messages [ 6615.436336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6616.396183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6617.242635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6618.002753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6618.866212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6619.626828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6620.329661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6621.338405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6622.285101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6623.137086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6624.093685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6624.909385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6625.839903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6626.831834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6627.881487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6628.720475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6629.855035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6630.786419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6631.812433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6632.622448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7118.124129] LustreError: 174066:0:(ofd_dev.c:1783:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 7118.138678] LustreError: 172261:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7864.828635] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 7864.832444] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7864.847717] LustreError: Skipped 1 previous similar message [ 7864.868594] Lustre: Skipped 20 previous similar messages [ 7864.887400] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7865.415707] Lustre: server umount lustre-OST0000 complete [ 7870.944706] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7870.956079] LustreError: Skipped 5 previous similar messages [ 7870.966940] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7872.995284] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7873.007283] Lustre: Skipped 1 previous similar message [ 7875.802554] Lustre: server umount lustre-MDT0000 complete [ 7878.131590] LustreError: 172242:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7878.153112] LustreError: 172242:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 7880.110880] LustreError: 172222:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787876331 with bad export cookie 18260252006819550847 [ 7880.115470] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7880.123798] LustreError: 172222:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7889.656788] Lustre: server umount lustre-MDT0001 complete [ 7901.714811] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing unload_modules_local [ 7904.731868] Key type lgssc unregistered [ 7905.130876] LNet: 227553:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7905.143382] LNetError: 227553:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7905.163730] LNet: Removed LNI 192.168.204.148@tcp [ 7906.232856] Key type .llcrypt unregistered [ 7906.237561] Key type ._llcrypt unregistered [ 7932.722772] Key type ._llcrypt registered [ 7932.729977] Key type .llcrypt registered [ 7932.824910] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 7933.955429] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7933.989407] alg: No test for adler32 (adler32-zlib) [ 7935.092833] Lustre: Lustre: Build Version: 2.17.57_99_gb2219ab [ 7935.441427] LNet: Added LNI 192.168.204.148@tcp [8/256/0/180] [ 7937.175202] Key type lgssc registered [ 7938.239760] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7945.440478] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7945.459485] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7946.821462] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7952.073430] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 7956.965550] LustreError: 228972:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7956.985081] LustreError: 228972:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 7961.231988] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7961.574112] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7967.209826] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 7977.907590] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7985.194171] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7992.936752] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7993.338932] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7999.406065] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 8009.042873] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8017.928921] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 8024.442100] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 8026.862431] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8026.892357] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 8026.900737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8026.915770] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 8026.920515] LustreError: 228991:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 8030.321033] Lustre: lustre-OST0000: Not available for connect from 192.168.204.48@tcp (stopping) [ 8032.234914] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8032.261161] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8032.280396] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8035.399686] Lustre: lustre-OST0000: Not available for connect from 192.168.204.48@tcp (stopping) [ 8035.411075] Lustre: Skipped 2 previous similar messages [ 8035.976190] Lustre: server umount lustre-OST0000 complete [ 8047.007148] LustreError: 228991:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 8072.203925] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8072.665618] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8073.720300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8073.739277] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 8079.860939] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 8084.010114] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8084.078175] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 8091.074495] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8093.988390] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 8103.373863] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8bfd8691a800.ost_server_uuid 50 [ 8104.999667] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8bfd8691a800.ost_server_uuid in FULL state after 0 sec [ 8109.553662] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8109.560889] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8109.576201] Lustre: Skipped 2 previous similar messages [ 8109.586295] Lustre: Skipped 1 previous similar message [ 8111.676756] Lustre: server umount lustre-OST0000 complete [ 8119.779616] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8119.787843] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8119.798241] Lustre: Skipped 1 previous similar message [ 8119.835549] Lustre: Skipped 1 previous similar message [ 8122.684771] Lustre: server umount lustre-MDT0000 complete [ 8124.897354] LustreError: 228967:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8124.942434] LustreError: 228967:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 8128.041721] LustreError: 228952:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787876579 with bad export cookie 6484007479393082630 [ 8128.047919] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8128.062891] LustreError: 228952:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8129.417148] Lustre: server umount lustre-MDT0001 complete [ 8142.913780] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing unload_modules_local [ 8145.920252] Key type lgssc unregistered [ 8146.199439] LNet: 234212:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8146.208982] LNetError: 234212:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8146.230140] LNet: Removed LNI 192.168.204.148@tcp [ 8147.050138] Key type .llcrypt unregistered [ 8147.052293] Key type ._llcrypt unregistered [ 8187.404607] Key type ._llcrypt registered [ 8187.410281] Key type .llcrypt registered [ 8187.621150] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 8189.106554] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8189.122563] alg: No test for adler32 (adler32-zlib) [ 8190.415964] Lustre: Lustre: Build Version: 2.17.57_99_gb2219ab [ 8190.695952] LNet: Added LNI 192.168.204.148@tcp [8/256/0/180] [ 8192.359653] Key type lgssc registered [ 8193.948678] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8204.239972] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8204.275308] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8205.849437] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8212.071550] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 8216.035359] LustreError: 235942:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8216.047095] LustreError: 235942:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 8221.153764] LustreError: 235941:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8222.694940] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8223.261669] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8229.014914] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 8232.626811] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 8239.149545] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8239.471548] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8245.405961] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 8250.731078] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 8255.123274] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8255.244427] Lustre: lustre-OST0001: new disk, initializing [ 8255.248255] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8255.252832] Lustre: 238254:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8255.368817] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8261.758944] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 8262.196622] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8262.205018] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8262.269168] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8274.456891] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8284.779280] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 8303.082747] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8303.087703] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8303.092872] Lustre: Skipped 1 previous similar message [ 8303.109526] Lustre: Skipped 3 previous similar messages [ 8308.197299] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8308.209220] Lustre: Skipped 3 previous similar messages [ 8313.312642] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8315.359152] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8315.541785] Lustre: server umount lustre-MDT0000 complete [ 8318.441101] LustreError: 235937:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8318.463867] LustreError: 235937:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [ 8318.910677] LustreError: 235922:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787876770 with bad export cookie 5956866637776006958 [ 8318.912876] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8318.917041] LustreError: 235922:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8319.328860] Lustre: server umount lustre-MDT0001 complete [ 8334.177578] Lustre: server umount lustre-OST0000 complete [ 8348.440267] Lustre: server umount lustre-OST0001 complete [ 8373.323890] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 8383.232383] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8383.763634] LustreError: 241365:0:(ldlm_lib.c:1228:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8383.869110] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8387.922082] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 8389.095795] LustreError: 241366:0:(ldlm_lib.c:1228:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8398.387492] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8398.990244] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8404.738222] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 8408.372448] Lustre: 242507:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8416.868598] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8417.319771] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8423.479474] LustreError: 242863:0:(ldlm_lib.c:1228:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8423.512813] LustreError: 242863:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 8423.938401] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 8429.568906] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49760) [ 8435.157302] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8440.825842] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 8442.263663] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 8451.691581] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8456.225395] Lustre: 244380:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8470.497250] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8470.516263] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8470.545523] Lustre: Skipped 2 previous similar messages [ 8470.577804] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8470.589579] Lustre: Skipped 3 previous similar messages [ 8471.534136] Lustre: mylustre-MDT0000-lwp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8471.553506] Lustre: Skipped 1 previous similar message [ 8476.164172] Lustre: server umount mylustre-MDT0000 complete [ 8476.644520] LustreError: 241365:0:(ldlm_lib.c:1228:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8476.680835] LustreError: 241365:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 8 previous similar messages [ 8480.691900] LustreError: 241341:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787876932 with bad export cookie 5956866637776009765 [ 8480.706633] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8480.710285] LustreError: 241341:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 8481.398537] Lustre: server umount mylustre-MDT0001 complete [ 8495.834709] Lustre: server umount mylustre-OST0000 complete [ 8497.695109] Lustre: 235203:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787876933/real 1787876933] req@ffff92c37b1cc3c0 x1874724537931648/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787876949 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8497.752764] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8501.038627] Lustre: server umount mylustre-OST0001 complete [ 8533.326276] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 8547.027285] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8547.604274] LustreError: 246688:0:(ldlm_lib.c:1228:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8547.714726] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8547.727415] Lustre: Skipped 1 previous similar message [ 8552.845184] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 8565.082181] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8571.435932] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 8574.851347] Lustre: 247833:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8582.686758] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8590.374399] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 8592.644672] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49792) [ 8603.954574] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8609.275930] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 8614.113390] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 8624.984671] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8629.314513] Lustre: 249704:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8639.978988] Lustre: tfs-MDT0000-lwp-OST0000: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8640.013597] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 8640.034465] Lustre: Skipped 7 previous similar messages [ 8646.127259] Lustre: server umount tfs-MDT0000 complete [ 8650.219663] LustreError: 247423:0:(ldlm_lib.c:1228:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8650.245883] LustreError: 247423:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 13 previous similar messages [ 8652.492815] LustreError: 246666:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787877104 with bad export cookie 5956866637776011655 [ 8652.505120] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8652.519598] LustreError: 246666:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8653.453548] Lustre: server umount tfs-MDT0001 complete [ 8671.594622] Lustre: server umount tfs-OST0000 complete [ 8671.716346] Lustre: 235204:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787877107/real 1787877107] req@ffff92c2618e9680 x1874724537991680/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787877123 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8671.784920] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8671.815638] Lustre: Skipped 3 previous similar messages [ 8674.291747] Lustre: 235205:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787877109/real 1787877109] req@ffff92c249b221c0 x1874724537991936/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787877125 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8677.107400] Lustre: server umount tfs-OST0001 complete [ 8701.667210] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 8715.180599] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8716.149737] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8716.164436] Lustre: Skipped 3 previous similar messages [ 8723.920753] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 8739.154659] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8744.478567] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 8747.747903] Lustre: 253100:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8755.091856] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8760.659197] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49824) [ 8762.393634] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 8771.205789] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8776.725242] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 8777.733730] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 8787.656276] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8792.896886] Lustre: 254972:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8800.882675] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 20:34:11 (1787877251) [ 8816.613917] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8816.622150] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8816.648784] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8816.662729] Lustre: Skipped 4 previous similar messages [ 8822.216759] Lustre: server umount lustre-MDT0000 complete [ 8822.757730] LustreError: 251958:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8822.783288] LustreError: 251958:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 13 previous similar messages [ 8825.996624] LustreError: 251934:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787877277 with bad export cookie 5956866637776013321 [ 8825.999077] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8826.008222] LustreError: 251934:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8832.535862] Lustre: server umount lustre-MDT0001 complete [ 8850.399914] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8850.507883] Lustre: server umount lustre-OST0000 complete [ 8860.942978] Lustre: server umount lustre-OST0001 complete [ 8868.313474] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_hostid [ 8879.152126] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 8921.426784] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8921.608853] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8921.630834] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8921.692158] Lustre: lustre-MDT0000: new disk, initializing [ 8921.753097] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 8921.758188] Lustre: Skipped 3 previous similar messages [ 8921.772544] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8925.406922] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 8935.331151] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8935.517929] Lustre: 258157:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8935.640450] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8935.656899] Lustre: Skipped 1 previous similar message [ 8935.764651] Lustre: lustre-MDT0001: new disk, initializing [ 8936.016813] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8936.037059] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8942.747327] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 8957.818759] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8958.080925] Lustre: lustre-OST0000: new disk, initializing [ 8958.084639] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8958.089403] Lustre: 259740:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8959.512741] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8959.540478] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8959.703599] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8966.821494] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 8981.527202] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8981.657925] Lustre: lustre-OST0001: new disk, initializing [ 8981.661585] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8981.666973] Lustre: 260614:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8983.437970] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8983.445748] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8983.564430] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8988.428526] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 9005.254784] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 20:37:35 (1787877455) [ 9022.546901] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 20:37:52 (1787877472) [ 9039.864879] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9039.886072] Lustre: Skipped 5 previous similar messages [ 9039.895253] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9039.901711] Lustre: Skipped 8 previous similar messages [ 9052.127181] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9052.579585] Lustre: server umount lustre-OST0000 complete [ 9058.788385] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9064.527633] Lustre: server umount lustre-MDT0000 complete [ 9070.372182] LustreError: 259750:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787877522 with bad export cookie 5956866637776014875 [ 9070.383335] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9070.390860] LustreError: 259750:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9075.687080] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9075.693088] Lustre: Skipped 9 previous similar messages [ 9076.849778] Lustre: server umount lustre-MDT0001 complete [ 9086.145331] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing unload_modules_local [ 9158.585492] Lustre: server umount lustre-OST0001 complete [ 9166.352657] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_hostid [ 9183.047193] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 9229.156637] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9229.546327] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9229.576482] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9229.637943] Lustre: lustre-MDT0000: new disk, initializing [ 9229.712541] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9229.719567] Lustre: Skipped 3 previous similar messages [ 9229.732456] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9234.964721] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 9247.436168] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9247.613146] Lustre: 265460:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9247.697905] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9247.708329] Lustre: Skipped 1 previous similar message [ 9247.804793] Lustre: lustre-MDT0001: new disk, initializing [ 9247.988791] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9248.008506] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9253.198611] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 9263.756901] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9270.801826] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9281.078266] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9281.627478] Lustre: lustre-OST0000: new disk, initializing [ 9281.640851] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9281.661673] Lustre: 267430:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9283.552628] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9283.564341] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9283.723026] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9290.768712] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 9304.713697] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9314.278374] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9314.281243] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9314.287316] LustreError: Skipped 1 previous similar message [ 9314.288138] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9314.342466] Lustre: Skipped 6 previous similar messages [ 9320.096702] Lustre: server umount lustre-OST0000 complete [ 9325.030126] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9331.626432] Lustre: server umount lustre-MDT0000 complete [ 9334.758928] LustreError: 265466:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9334.771535] LustreError: 265466:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 14 previous similar messages [ 9336.304150] LustreError: 265452:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787877788 with bad export cookie 5956866637776016345 [ 9336.307342] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9336.314200] LustreError: 265452:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 9336.720381] Lustre: server umount lustre-MDT0001 complete [ 9350.268488] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing unload_modules_local [ 9353.491825] Key type lgssc unregistered [ 9353.841216] LNet: 269194:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9353.848433] LNetError: 269194:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9353.866103] LNet: Removed LNI 192.168.204.148@tcp [ 9355.025190] Key type .llcrypt unregistered [ 9355.027952] Key type ._llcrypt unregistered [ 9383.401881] Key type ._llcrypt registered [ 9383.404472] Key type .llcrypt registered [ 9383.580505] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 9384.618425] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9384.685962] alg: No test for adler32 (adler32-zlib) [ 9386.037751] Lustre: Lustre: Build Version: 2.17.57_99_gb2219ab [ 9386.405272] LNet: Added LNI 192.168.204.148@tcp [8/256/0/180] [ 9388.127305] Key type lgssc registered [ 9389.945884] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9399.019841] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9399.051518] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9400.716853] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9405.407946] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 9411.047333] LustreError: 270660:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9411.075442] LustreError: 270660:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 9414.587544] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9414.882968] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9419.399756] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 9428.542853] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9436.287560] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9445.457264] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9445.814277] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9446.843458] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 9453.222065] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 9464.545488] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9465.878190] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 9470.437576] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9470.440962] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9470.470196] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9470.487158] Lustre: Skipped 1 previous similar message [ 9475.554145] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9480.677972] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9480.688024] Lustre: Skipped 2 previous similar messages [ 9482.207152] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9482.555768] Lustre: server umount lustre-OST0000 complete [ 9490.912322] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9490.935587] Lustre: Skipped 1 previous similar message [ 9490.942111] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9493.550231] Lustre: server umount lustre-MDT0000 complete [ 9496.034359] LustreError: 271388:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9496.069769] LustreError: 271388:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 9497.722657] LustreError: 270643:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787877949 with bad export cookie 18113042188458396686 [ 9497.733889] LustreError: 270643:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9497.737763] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9498.137172] Lustre: server umount lustre-MDT0001 complete [ 9510.164518] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing unload_modules_local [ 9513.620806] Key type lgssc unregistered [ 9514.019815] LNet: 274250:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9514.032835] LNetError: 274250:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9514.057284] LNet: Removed LNI 192.168.204.148@tcp [ 9515.000702] Key type .llcrypt unregistered [ 9515.004327] Key type ._llcrypt unregistered [ 9534.056475] Key type ._llcrypt registered [ 9534.063703] Key type .llcrypt registered [ 9534.217892] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 9535.232513] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9535.244774] alg: No test for adler32 (adler32-zlib) [ 9536.370442] Lustre: Lustre: Build Version: 2.17.57_99_gb2219ab [ 9536.641742] LNet: Added LNI 192.168.204.148@tcp [8/256/0/180] [ 9538.303192] Key type lgssc registered [ 9539.987434] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9555.342043] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9555.420452] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9571.167432] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 9576.287529] LustreError: 275447:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.148@tcp: failed processing log, type 4: rc = -110 [ 9602.975200] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9610.489754] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 9618.811146] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9621.544267] Lustre: server umount lustre-OST0000 complete [ 9642.737659] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing unload_modules_local [ 9646.280489] Key type lgssc unregistered [ 9646.897284] LNet: 277083:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9646.913133] LNetError: 277083:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9646.934877] LNet: Removed LNI 192.168.204.148@tcp [ 9648.028091] Key type .llcrypt unregistered [ 9648.034937] Key type ._llcrypt unregistered [ 9706.889387] Key type ._llcrypt registered [ 9706.891566] Key type .llcrypt registered [ 9706.972852] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 9708.423243] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9708.459614] alg: No test for adler32 (adler32-zlib) [ 9709.563364] Lustre: Lustre: Build Version: 2.17.57_99_gb2219ab [ 9709.863157] LNet: Added LNI 192.168.204.148@tcp [8/256/0/180] [ 9711.551272] Key type lgssc registered [ 9712.924612] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9729.098251] loop0: detected capacity change from 0 to 204800000 [ 9729.419177] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9729.435225] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9729.645983] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9730.871899] loop0: detected capacity change from 0 to 204800000 [ 9731.100651] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9731.127204] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9731.796758] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9732.857149] loop0: detected capacity change from 0 to 204800000 [ 9733.259216] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9733.281213] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9733.532037] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9735.421460] loop0: detected capacity change from 0 to 204800000 [ 9735.703585] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9735.744738] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9736.086446] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9738.722455] loop0: detected capacity change from 0 to 204800000 [ 9738.809793] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9745.401416] loop0: detected capacity change from 0 to 204800000 [ 9745.468857] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9750.531075] loop0: detected capacity change from 0 to 204800000 [ 9750.565987] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9755.389745] loop0: detected capacity change from 0 to 204800000 [ 9755.417480] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9760.321571] loop0: detected capacity change from 0 to 204800000 [ 9760.442337] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9760.509371] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9760.624802] Lustre: 280504:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9760.635976] Lustre: 280504:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 9760.655101] Lustre: 280504:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9760.681924] Lustre: 280504:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9760.701744] Lustre: 280504:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 9760.721584] Lustre: 280504:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9760.739544] Lustre: 280504:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 9760.768478] Lustre: 280504:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 9760.801800] Lustre: 280504:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 9760.819817] Lustre: 280504:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 9760.845224] Lustre: 280504:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 9760.859240] Lustre: 280504:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 9760.877426] Lustre: 280504:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 9760.886862] Lustre: 280504:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 9766.747091] Lustre: server umount MGS complete [ 9767.703922] loop0: detected capacity change from 0 to 204800000 [ 9767.774028] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9768.442739] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9768.474349] Lustre: lustre-MDT0000: Aborting client recovery [ 9769.732901] loop1: detected capacity change from 0 to 204800000 [ 9769.938128] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9770.141533] Lustre: 280935:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9770.170887] Lustre: 280935:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9770.207103] Lustre: 280935:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9770.245482] Lustre: 280935:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 9770.260667] Lustre: 280935:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9770.285280] Lustre: 280935:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 9770.309834] Lustre: 280935:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 9770.360057] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9770.579495] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9770.646596] Lustre: lustre-MDT0001: Aborting client recovery [ 9771.637319] loop2: detected capacity change from 0 to 204800000 [ 9771.778323] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9771.884519] Lustre: 281014:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9771.901769] Lustre: 281014:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9771.916102] Lustre: 281014:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9771.925449] Lustre: 281014:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9771.939533] Lustre: 281014:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9771.951341] Lustre: 281014:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9771.960746] Lustre: 281014:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9772.081633] Lustre: Found index 0 for lustre-OST0000, updating log [ 9772.114313] Lustre: 281033:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9772.186700] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9772.219643] Lustre: lustre-OST0000: Aborting recovery [ 9773.401570] loop3: detected capacity change from 0 to 204800000 [ 9773.656955] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9773.763270] Lustre: 281098:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9773.775728] Lustre: 281098:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9773.784921] Lustre: 281098:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9773.793660] Lustre: 281098:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9773.804398] Lustre: 281098:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9773.823917] Lustre: 281098:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9773.843661] Lustre: 281098:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9773.915140] Lustre: Found index 1 for lustre-OST0001, updating log [ 9774.018262] Lustre: 281108:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9774.100440] Lustre: lustre-OST0001: Aborting recovery [ 9779.197930] LustreError: 280879:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 9780.197916] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9780.208905] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9780.262100] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 9780.277040] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 9781.229086] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 9781.234122] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 9783.091390] LustreError: 280846:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 7959us: rc = -2 [ 9783.611240] LustreError: 280844:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 4416us: rc = -2 [ 9783.657380] LustreError: 280844:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 15 previous similar messages [ 9784.621596] LustreError: 280844:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 2415us: rc = -2 [ 9784.654957] LustreError: 280844:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 23 previous similar messages [ 9801.184948] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9801.199312] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9801.225346] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9801.702715] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9801.713063] Lustre: Skipped 2 previous similar messages [ 9805.250661] Lustre: server umount lustre-MDT0000 complete [ 9806.820380] LustreError: 280843:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9806.884379] LustreError: 280843:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 5 previous similar messages [ 9806.921698] LustreError: 280829:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787878258 with bad export cookie 6479520307831182121 [ 9806.925834] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9806.944190] LustreError: 280829:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9811.936563] LustreError: 281500:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9811.936805] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9811.937848] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9811.937853] Lustre: Skipped 3 previous similar messages [ 9811.961845] LustreError: 281500:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 9812.012481] Lustre: Skipped 1 previous similar message [ 9813.611657] Lustre: server umount lustre-MDT0001 complete [ 9821.514102] Lustre: server umount lustre-OST0000 complete [ 9828.828797] Lustre: server umount lustre-OST0001 complete [ 9838.964995] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 20:51:28 (1787878288) [ 9871.207912] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_hostid [ 9882.805183] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [ 9926.324164] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9926.771131] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9926.821517] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9926.913767] Lustre: lustre-MDT0000: new disk, initializing [ 9926.992744] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9926.997510] Lustre: Skipped 1 previous similar message [ 9927.011208] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9933.117300] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 9945.451853] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9945.618338] Lustre: 285181:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9945.681710] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9945.696249] Lustre: Skipped 1 previous similar message [ 9945.835895] Lustre: lustre-MDT0001: new disk, initializing [ 9945.979199] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9946.020368] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9946.050614] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9951.405784] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 9962.139278] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9968.766520] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9976.330081] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9976.627967] Lustre: lustre-OST0000: new disk, initializing [ 9976.634173] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9976.643562] Lustre: 287150:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9976.798086] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9983.077299] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [ 9983.568862] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9983.592269] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9983.712086] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9993.180611] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9999.075960] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10013.800342] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10018.112358] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10023.933467] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [10081.256920] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10081.264050] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10081.268288] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10081.268297] Lustre: Skipped 1 previous similar message [10081.276735] Lustre: Skipped 1 previous similar message [10086.380477] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10086.405892] Lustre: Skipped 2 previous similar messages [10091.497330] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10091.504247] Lustre: Skipped 1 previous similar message [10093.023277] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10093.304801] Lustre: server umount lustre-OST0000 complete [10099.682882] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10099.696461] LustreError: Skipped 1 previous similar message [10099.699521] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10099.715531] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10104.981615] Lustre: server umount lustre-MDT0000 complete [10106.862423] LustreError: 285186:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10106.881381] LustreError: 285186:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [10109.949356] LustreError: 285976:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787878561 with bad export cookie 6479520307831184928 [10109.953942] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10109.967713] LustreError: 285976:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10110.395326] Lustre: server umount lustre-MDT0001 complete [10121.058674] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10126.559595] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [10132.604731] Lustre: server umount MGS complete [10146.474698] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10146.973791] LustreError: 291053:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10147.000611] LustreError: 291065:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10147.086458] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10151.736866] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [10152.418159] LustreError: 291052:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10152.437057] LustreError: 291052:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [10161.086641] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10166.079734] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [10174.249767] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10180.249953] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10188.147506] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10188.464189] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10188.469633] Lustre: Skipped 1 previous similar message [10195.817994] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [10204.803792] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10237.921694] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10237.922320] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10237.943836] LustreError: Skipped 1 previous similar message [10237.945983] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10237.945991] Lustre: Skipped 2 previous similar messages [10237.996965] Lustre: Skipped 2 previous similar messages [10250.208154] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10250.410437] Lustre: server umount lustre-OST0000 complete [10256.366373] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10256.385556] Lustre: Skipped 1 previous similar message [10269.151728] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10269.438320] Lustre: server umount lustre-MDT0000 complete [10271.715392] LustreError: 291052:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10271.751243] LustreError: 291052:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [10273.921634] LustreError: 291034:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787878725 with bad export cookie 6479520307831186867 [10273.933163] LustreError: 291034:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10273.937936] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10274.315384] Lustre: server umount lustre-MDT0001 complete [10287.600773] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing unload_modules_local [10290.549934] Key type lgssc unregistered [10290.879125] LNet: 295005:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10290.884639] LNetError: 295005:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10290.900582] LNet: Removed LNI 192.168.204.148@tcp [10291.866574] Key type .llcrypt unregistered [10291.869076] Key type ._llcrypt unregistered [10334.511252] Key type ._llcrypt registered [10334.516315] Key type .llcrypt registered [10334.739619] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_hostid [10354.209777] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [10355.806226] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10355.951046] alg: No test for adler32 (adler32-zlib) [10357.406576] Lustre: Lustre: Build Version: 2.17.57_99_gb2219ab [10357.742417] LNet: Added LNI 192.168.204.148@tcp [8/256/0/180] [10359.561303] Key type lgssc registered [10360.686382] Lustre: Echo OBD driver; http://www.lustre.org/ [10399.151544] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10399.204066] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10400.409332] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10400.476712] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10400.606232] Lustre: lustre-MDT0000: new disk, initializing [10400.828806] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10400.845768] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10405.467289] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [10417.781887] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10418.012282] Lustre: 298965:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10418.061828] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10418.065649] Lustre: Skipped 1 previous similar message [10418.119402] Lustre: lustre-MDT0001: new disk, initializing [10418.260939] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10418.330243] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10418.352911] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10423.438455] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [10432.982327] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10439.384453] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10446.901200] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10447.263226] Lustre: lustre-OST0000: new disk, initializing [10447.270627] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10447.285541] Lustre: 300946:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10447.429216] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10455.698087] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10455.717106] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10455.995673] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10456.792356] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [10470.283172] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10477.220891] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10486.916904] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10490.964752] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10496.677539] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [10537.965253] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10537.970661] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10538.010390] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10538.029464] Lustre: Skipped 1 previous similar message [10538.049275] Lustre: Skipped 1 previous similar message [10541.848163] Lustre: server umount lustre-OST0000 complete [10548.195650] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10548.207652] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10548.221528] Lustre: Skipped 1 previous similar message [10548.252880] Lustre: Skipped 1 previous similar message [10553.312440] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10553.339997] Lustre: Skipped 1 previous similar message [10554.614433] Lustre: server umount lustre-MDT0000 complete [10559.031292] LustreError: 298957:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787879010 with bad export cookie 14042824723876576350 [10559.037317] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10559.040282] LustreError: 298957:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10559.470332] Lustre: server umount lustre-MDT0001 complete [10568.360492] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10572.739620] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [10577.522739] Lustre: server umount MGS complete [10589.351115] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10589.963662] LustreError: 304714:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10589.984553] LustreError: 304714:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [10589.989741] LustreError: 304727:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10590.055937] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10594.970356] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [10595.308384] LustreError: 304715:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10595.331934] LustreError: 304715:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [10599.397872] LustreError: 304714:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10603.206147] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10603.633569] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10607.981267] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [10615.422798] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10620.755096] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10627.856392] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10628.252987] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10635.022889] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [10643.727746] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10675.681766] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10675.690833] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10675.703517] Lustre: Skipped 1 previous similar message [10675.726379] Lustre: Skipped 1 previous similar message [10678.885871] Lustre: server umount lustre-OST0000 complete [10685.413600] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10685.427396] LustreError: Skipped 1 previous similar message [10685.432262] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10685.449139] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10689.234426] Lustre: server umount lustre-MDT0000 complete [10691.046124] LustreError: 305444:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10691.077618] LustreError: 305444:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [10693.002457] LustreError: 304694:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787879144 with bad export cookie 14042824723876578177 [10693.011842] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10693.017210] LustreError: 304694:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10693.445872] Lustre: server umount lustre-MDT0001 complete [10705.225919] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing unload_modules_local [10708.124465] Key type lgssc unregistered [10708.476107] LNet: 308662:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10708.487779] LNetError: 308662:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10708.524310] LNet: Removed LNI 192.168.204.148@tcp [10709.381273] Key type .llcrypt unregistered [10709.383389] Key type ._llcrypt unregistered [10731.016894] Key type ._llcrypt registered [10731.022018] Key type .llcrypt registered [10731.149785] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_hostid [10745.427992] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [10746.715272] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10746.749092] alg: No test for adler32 (adler32-zlib) [10748.271879] Lustre: Lustre: Build Version: 2.17.57_99_gb2219ab [10748.607947] LNet: Added LNI 192.168.204.148@tcp [8/256/0/180] [10750.319179] Key type lgssc registered [10752.157676] Lustre: Echo OBD driver; http://www.lustre.org/ [10783.611665] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [10799.584493] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10799.627568] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10800.873956] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10800.909054] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10800.984801] Lustre: lustre-MDT0000: new disk, initializing [10801.052889] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10801.081836] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10806.626947] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [10823.111357] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10823.754624] Lustre: lustre-OST0000: new disk, initializing [10823.760503] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10823.775155] Lustre: Skipped 1 previous similar message [10823.791865] Lustre: 312717:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10824.053092] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10833.690238] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [10833.716231] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [10833.809632] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [10834.949903] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [11786.211256] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11786.232381] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11787.145948] Lustre: server umount lustre-MDT0000 complete [11790.438528] LustreError: 311902:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787880242 with bad export cookie 1040406811369044846 [11790.447378] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11790.449258] LustreError: 311902:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11792.649450] Lustre: server umount lustre-OST0000 complete [11819.635961] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_hostid [11829.826977] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [11865.665958] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11866.044384] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11866.117801] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11866.205401] Lustre: lustre-MDT0000: new disk, initializing [11866.296837] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11866.318293] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11871.701504] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [11882.715528] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11882.859676] Lustre: 317072:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11882.950633] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11882.955528] Lustre: Skipped 1 previous similar message [11883.047508] Lustre: lustre-MDT0001: new disk, initializing [11883.188545] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11883.233039] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11883.247017] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11888.265453] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [11898.409679] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11904.706564] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11915.677644] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11916.172362] Lustre: lustre-OST0000: new disk, initializing [11916.180835] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11916.194940] Lustre: 319041:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11916.284559] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11926.089691] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11926.106852] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11926.287799] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11926.669438] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [11938.571617] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11944.234460] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11944.477980] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11948.957250] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11949.150284] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11952.105836] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11952.109208] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11952.113717] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11952.138503] Lustre: Skipped 1 previous similar message [11957.347455] Lustre: server umount lustre-OST0000 complete [11962.345389] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11962.359675] Lustre: Skipped 1 previous similar message [11962.367815] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11962.382131] Lustre: Skipped 2 previous similar messages [11967.442610] Lustre: server umount lustre-MDT0000 complete [11967.469639] LustreError: 318233:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11967.482894] LustreError: 318233:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 4 previous similar messages [11971.007077] LustreError: 319040:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787880422 with bad export cookie 1040406811371985665 [11971.020715] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11971.029059] LustreError: 319040:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11971.530507] Lustre: server umount lustre-MDT0001 complete [11975.068352] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [11976.811930] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 21:27:07 (1787880427) [11983.128129] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11983.520671] LustreError: 321354:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11983.571237] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11988.026786] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [11988.960491] LustreError: 321355:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11994.082641] LustreError: 321354:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11998.097995] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11998.639706] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12004.450735] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [12013.368978] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12020.001839] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12028.342635] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12028.885441] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12036.705625] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [12043.937716] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12051.981306] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12052.207387] Lustre: lustre-OST0001: new disk, initializing [12052.214632] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [12052.222206] Lustre: 324193:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12052.394179] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12059.366698] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [12060.253076] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [12060.263380] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [12060.329274] LustreError: 324588:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12060.354332] LustreError: 324588:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [12061.357194] LustreError: 324597:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [12062.436679] LustreError: 324588:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12062.445111] LustreError: 324588:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [12064.493100] LustreError: 324597:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [12064.506883] LustreError: 324597:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [12068.584899] LustreError: 324588:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12068.621228] LustreError: 324588:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 6 previous similar messages [12071.689055] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12071.909209] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12076.849775] LustreError: 324588:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12076.868728] LustreError: 324588:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [12078.743922] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8bfd8567e800.ost_server_uuid 50 [12080.832913] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8bfd8567e800.ost_server_uuid in FULL state after 0 sec [12087.398810] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8bfd8567e800.ost_server_uuid 50 [12089.427794] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8bfd8567e800.ost_server_uuid in IDLE state after 0 sec [12091.408769] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [12106.731715] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12106.735473] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12106.760265] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12106.791597] Lustre: Skipped 1 previous similar message [12111.853377] LustreError: 323179:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12111.867944] LustreError: 323179:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [12111.987391] Lustre: server umount lustre-OST0001 complete [12120.032252] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12120.043691] LustreError: Skipped 1 previous similar message [12120.059480] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12120.089850] Lustre: Skipped 1 previous similar message [12120.111210] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12120.131977] Lustre: Skipped 1 previous similar message [12122.083645] LustreError: 324589:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12122.130405] LustreError: 324589:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 4 previous similar messages [12126.083415] Lustre: server umount lustre-OST0000 complete [12131.808806] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12131.827415] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12131.853468] Lustre: Skipped 1 previous similar message [12131.859135] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12131.862896] Lustre: Skipped 2 previous similar messages [12136.915437] Lustre: server umount lustre-MDT0000 complete [12141.222834] LustreError: 321336:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787880592 with bad export cookie 1040406811371986547 [12141.228620] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12141.242890] LustreError: 321336:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12141.933613] Lustre: server umount lustre-MDT0001 complete [12156.915682] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing unload_modules_local [12161.294484] Key type lgssc unregistered [12161.795793] LNet: 326891:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12161.803123] LNetError: 326891:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12161.839476] LNet: Removed LNI 192.168.204.148@tcp [12163.223811] Key type .llcrypt unregistered [12163.227840] Key type ._llcrypt unregistered [12193.722189] Key type ._llcrypt registered [12193.730043] Key type .llcrypt registered [12193.876153] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [12195.083640] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12195.114810] alg: No test for adler32 (adler32-zlib) [12196.195722] Lustre: Lustre: Build Version: 2.17.57_99_gb2219ab [12196.469695] LNet: Added LNI 192.168.204.148@tcp [8/256/0/180] [12198.175679] Key type lgssc registered [12199.812930] Lustre: Echo OBD driver; http://www.lustre.org/ [12211.432438] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12211.498242] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12213.311869] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12219.261916] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [12222.436262] LustreError: 328313:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12222.456817] LustreError: 328313:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [12227.556357] LustreError: 328314:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12228.405909] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12232.674021] LustreError: 328313:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12237.302512] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12237.814817] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12243.194910] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [12254.610306] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12254.921447] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12259.067463] LustreError: 329994:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12259.104517] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [12261.014551] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [12263.144366] LustreError: 329994:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12263.171988] LustreError: 329994:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [12268.814541] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12273.657921] LustreError: 329994:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12273.687433] LustreError: 329994:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [12276.456102] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12276.774169] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12281.872338] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [12283.945617] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [12293.069980] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12302.040941] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [12302.506619] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12420.067243] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12420.075512] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12420.081203] Lustre: Skipped 1 previous similar message [12425.189897] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12425.198635] Lustre: Skipped 2 previous similar messages [12430.307395] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12430.314140] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12430.331205] Lustre: Skipped 1 previous similar message [12430.450399] Lustre: server umount lustre-OST0001 complete [12435.428977] LustreError: 329994:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12437.475218] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12437.481432] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12437.496795] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12438.497514] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12438.508901] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12445.666461] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12445.684290] Lustre: Skipped 4 previous similar messages [12450.271238] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12450.482222] Lustre: server umount lustre-OST0000 complete [12455.906137] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12455.919914] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12455.924272] Lustre: Skipped 1 previous similar message [12469.215115] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12469.332201] Lustre: server umount lustre-MDT0000 complete [12471.267517] LustreError: 328313:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12471.288863] LustreError: 328313:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 6 previous similar messages [12473.955866] LustreError: 328294:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787880925 with bad export cookie 6127199354633557061 [12473.958087] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12473.973080] LustreError: 328294:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12474.410115] Lustre: server umount lustre-MDT0001 complete [12486.932854] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing unload_modules_local [12490.734643] Key type lgssc unregistered [12491.014918] LNet: 333294:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12491.019645] LNetError: 333294:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12491.034080] LNet: Removed LNI 192.168.204.148@tcp [12492.110914] Key type .llcrypt unregistered [12492.113840] Key type ._llcrypt unregistered [12547.005581] Key type ._llcrypt registered [12547.009676] Key type .llcrypt registered [12547.160679] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [12548.205952] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12548.221305] alg: No test for adler32 (adler32-zlib) [12549.538927] Lustre: Lustre: Build Version: 2.17.57_99_gb2219ab [12549.843312] LNet: Added LNI 192.168.204.148@tcp [8/256/0/180] [12551.543211] Key type lgssc registered [12552.803134] Lustre: Echo OBD driver; http://www.lustre.org/ [12564.146888] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12564.166137] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12565.612575] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12571.020357] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [12576.227374] LustreError: 335585:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12576.246756] LustreError: 335585:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [12581.354419] LustreError: 335584:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12583.610854] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12583.993984] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12589.569831] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [12593.677111] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [12602.293481] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12602.797988] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12608.938717] LustreError: 337076:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12610.995807] LustreError: 337077:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12611.008308] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [12611.437156] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [12616.171918] LustreError: 337339:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12616.199879] LustreError: 337339:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [12622.386974] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12622.694144] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12627.975295] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [12631.253244] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [12641.989627] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12649.902558] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [12665.826109] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12665.828852] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12665.839916] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12668.901550] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12668.915925] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12668.961742] Lustre: Skipped 2 previous similar messages [12671.119876] Lustre: server umount lustre-MDT0000 complete [12674.034178] LustreError: 335580:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12674.057491] LustreError: 335580:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 5 previous similar messages [12675.718822] LustreError: 335567:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787881127 with bad export cookie 18152209048531445857 [12675.732606] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12675.739327] LustreError: 335567:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12676.217146] Lustre: server umount lustre-MDT0001 complete [12690.374326] Lustre: server umount lustre-OST0000 complete [12694.498201] Lustre: 334849:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787881130/real 1787881130] req@ffff92c35e359a40 x1874729108576768/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787881146 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [12694.572516] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12695.532352] Lustre: 334851:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787881131/real 1787881131] req@ffff92c35e35bc00 x1874729108577024/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787881147 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [12696.538658] Lustre: server umount lustre-OST0001 complete [12719.794296] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [12735.253321] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12735.729928] LustreError: 340852:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12735.864435] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12741.154399] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [12752.127608] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12752.583344] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12758.041498] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [12761.594675] Lustre: 341994:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [12768.536685] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12768.857992] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12775.312920] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [12778.093034] LustreError: 342349:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12778.112328] LustreError: 342349:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [12781.166748] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [12784.781509] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12790.302202] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [12791.468316] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [12798.676702] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12801.812117] Lustre: 343863:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [12808.675379] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12808.684405] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12808.709145] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12808.719835] Lustre: Skipped 3 previous similar messages [12810.723422] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12812.330595] Lustre: server umount lustre-MDT0000 complete [12817.469641] LustreError: 340833:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787881269 with bad export cookie 18152209048531447558 [12817.486533] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12817.941290] Lustre: server umount lustre-MDT0001 complete [12833.097355] Lustre: server umount lustre-OST0000 complete [12836.319240] Lustre: 334848:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787881272/real 1787881272] req@ffff92c3431e21c0 x1874729108627072/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787881288 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [12836.377568] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12836.412253] Lustre: Skipped 3 previous similar messages [12838.172257] Lustre: server umount lustre-OST0001 complete [12843.665235] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [12845.554405] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [12848.191747] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 21:41:37 (1787881297) [12871.968782] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [12881.549481] loop0: detected capacity change from 0 to 1099511627776 [12882.293398] SGI XFS with ACLs, security attributes, quota, no debug enabled [12882.304229] *** VALIDATE xfs *** [12882.326777] XFS (loop0): Mounting V5 Filesystem [12882.392926] XFS (loop0): Ending clean mount [12883.875619] loop1: detected capacity change from 0 to 18691697672192 [12996.213354] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_hostid [13007.746471] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [13050.923838] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13051.122881] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13051.150823] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13051.215344] Lustre: lustre-MDT0000: new disk, initializing [13051.280790] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13051.289094] Lustre: Skipped 1 previous similar message [13051.309687] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13057.449788] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [13069.159854] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13069.276486] Lustre: 349000:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13069.376800] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13069.383548] Lustre: Skipped 1 previous similar message [13069.475428] Lustre: lustre-MDT0001: new disk, initializing [13069.633496] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13069.648728] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13074.239165] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [13083.928817] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13090.678577] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13098.895236] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13099.193213] Lustre: lustre-OST0000: new disk, initializing [13099.198407] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13099.211875] Lustre: 350972:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13101.014168] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13101.034476] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13101.211490] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13105.966499] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [13116.789706] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13122.021470] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [13122.225618] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [13126.206832] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [13126.414567] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [13131.753624] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13131.761309] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [13131.793520] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13131.793532] Lustre: Skipped 3 previous similar messages [13134.746315] Lustre: server umount lustre-OST0000 complete [13141.476589] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13141.494755] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13141.502855] Lustre: Skipped 1 previous similar message [13145.723824] Lustre: server umount lustre-MDT0000 complete [13147.134717] LustreError: 349013:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13147.167797] LustreError: 349013:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 7 previous similar messages [13151.120311] LustreError: 348994:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787881602 with bad export cookie 18152209048531449098 [13151.122478] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13151.132379] LustreError: 348994:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [13151.657855] Lustre: server umount lustre-MDT0001 complete [13162.097787] XFS (loop0): Unmounting Filesystem [13167.150077] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 21:46:57 (1787881617) [13174.434429] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13174.957373] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13174.967051] Lustre: Skipped 2 previous similar messages [13179.378562] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [13188.675042] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13193.556292] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [13201.481677] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13207.941438] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13217.786147] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13224.725504] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [13233.492893] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13243.875705] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13243.878863] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13243.889783] Lustre: Skipped 4 previous similar messages [13243.900498] Lustre: Skipped 2 previous similar messages [13249.350257] Lustre: server umount lustre-OST0000 complete [13260.088718] Lustre: server umount lustre-MDT0000 complete [13265.008709] LustreError: 355378:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787881716 with bad export cookie 18152209048531449987 [13265.021450] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13265.035783] LustreError: 355378:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [13265.530623] Lustre: server umount lustre-MDT0001 complete [13279.082707] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing unload_modules_local [13281.895084] Key type lgssc unregistered [13282.171578] LNet: 357169:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13282.183638] LNetError: 357169:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13283.237753] LNet: Removed LNI 192.168.204.148@tcp [13284.507195] Key type .llcrypt unregistered [13284.509511] Key type ._llcrypt unregistered [13316.643801] Key type ._llcrypt registered [13316.647755] Key type .llcrypt registered [13316.837473] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [13318.558920] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [13318.593313] alg: No test for adler32 (adler32-zlib) [13319.686989] Lustre: Lustre: Build Version: 2.17.57_99_gb2219ab [13320.053562] LNet: Added LNI 192.168.204.148@tcp [8/256/0/180] [13321.767192] Key type lgssc registered [13323.119865] Lustre: Echo OBD driver; http://www.lustre.org/ [13331.888613] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [13331.926721] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13333.559584] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13338.478727] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [13343.717261] LustreError: 358741:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13343.740027] LustreError: 358741:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [13348.835170] LustreError: 358740:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13350.778323] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13351.672210] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13358.583981] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [13367.589863] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13374.366159] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13382.908926] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13383.377467] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13391.200448] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [13399.964356] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13413.344612] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13413.355579] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13413.396247] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13414.375922] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13414.376316] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13414.421283] Lustre: Skipped 1 previous similar message [13417.585809] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.48@tcp (stopping) [13418.882378] Lustre: server umount lustre-MDT0001 complete [13419.501217] LustreError: 358740:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13419.522030] LustreError: 358740:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [13422.665460] LustreError: 358736:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.48@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13427.799684] LustreError: 359837:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.48@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13427.844044] LustreError: 359837:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [13431.406590] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13431.533276] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13431.539728] Lustre: Found index 1 for lustre-MDT0001, updating log [13431.595552] Lustre: 358729:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13431.887641] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13436.857296] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [13436.919832] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13438.449094] LustreError: 362260:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13438.456151] LustreError: 362260:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13438.486362] LustreError: 358732:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13438.550878] LustreError: 362264:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.204.148@tcp: cfg command failed: rc = -17 [13438.567232] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [13438.567232] [13447.137700] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13447.169166] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13447.179216] Lustre: Skipped 1 previous similar message [13452.257395] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13452.268759] Lustre: Skipped 2 previous similar messages [13452.545464] Lustre: server umount lustre-MDT0001 complete [13453.382658] LustreError: 358737:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.48@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13453.403175] LustreError: 358737:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [13464.106535] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13464.194457] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13464.201358] Lustre: Found index 1 for lustre-MDT0001, updating log [13464.261046] Lustre: 358729:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13464.520504] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13468.980581] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [13469.686307] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13470.718414] LustreError: 363340:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13470.730647] LustreError: 363340:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13470.754921] LustreError: 358732:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13470.765586] LustreError: 358732:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13470.880508] LustreError: 363349:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [13470.894057] LustreError: 363349:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [13470.910348] LustreError: 363349:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.204.148@tcp: cfg command failed: rc = -17 [13470.921908] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.204.148@tcp [13470.921908] [13775.968180] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.48@tcp (stopping) [13776.867910] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13776.870569] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13776.891272] Lustre: Skipped 1 previous similar message [13781.924664] Lustre: server umount lustre-MDT0001 complete [13781.989827] LustreError: 358740:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13782.009337] LustreError: 358740:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 7 previous similar messages [13792.514975] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13792.696225] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13792.705614] Lustre: Found index 1 for lustre-MDT0001, updating log [13792.775445] Lustre: 358728:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13793.173524] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13798.060924] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [13798.385912] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13799.962939] LustreError: 371672:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13799.973325] LustreError: 371672:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13799.995179] LustreError: 358732:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13800.009275] LustreError: 358732:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13800.086193] LustreError: 371680:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [13800.116364] LustreError: 371680:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.204.148@tcp: cfg command failed: rc = -114 [13800.124770] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [13800.124770] [14120.933084] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14120.934819] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [14120.935587] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14120.935593] Lustre: Skipped 3 previous similar messages [14120.941150] Lustre: Skipped 3 previous similar messages [14124.073668] Lustre: server umount lustre-MDT0000 complete [14126.054339] LustreError: 358736:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14126.078742] LustreError: 358736:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 8 previous similar messages [14127.685753] LustreError: 358719:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787882579 with bad export cookie 1264939768676730258 [14127.692068] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14127.707016] LustreError: 358719:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [14131.168763] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14134.155657] Lustre: server umount lustre-MDT0001 complete [14137.941475] Lustre: server umount lustre-OST0000 complete [14146.741889] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_hostid [14157.679852] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [14193.799845] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 22:04:04 (1787882644) [14200.525803] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14200.726583] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [14200.750464] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [14200.809323] Lustre: lustre-MDT0000: new disk, initializing [14200.883460] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [14200.898360] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [14205.725882] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [14216.654576] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14216.765339] Lustre: 382339:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [14216.829221] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [14216.835312] Lustre: Skipped 1 previous similar message [14216.944717] Lustre: lustre-MDT0001: new disk, initializing [14217.058356] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [14217.101958] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [14217.115954] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [14221.097119] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [14230.326273] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [14237.190056] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [14244.971532] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14245.323476] Lustre: lustre-OST0000: new disk, initializing [14245.328426] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [14245.336437] Lustre: 384306:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [14245.421632] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [14251.490223] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_default_debug -1 all [14252.619724] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [14252.630873] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [14252.696427] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [14261.353832] Lustre: DEBUG MARKER: oleg448-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [14269.926343] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [14269.934438] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14269.961468] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14269.970032] Lustre: Skipped 4 previous similar messages [14270.836109] Lustre: server umount lustre-MDT0000 complete [14273.505710] LustreError: 385113:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14273.542451] LustreError: 385113:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 4 previous similar messages [14274.663063] LustreError: 382329:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787882726 with bad export cookie 1264939768676732211 [14274.668494] LustreError: MGC192.168.204.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14274.684420] LustreError: 382329:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [14274.693932] LustreError: 383144:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [14274.694207] LustreError: 383144:0:(client.c:1394:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@00000000cb1eac2b x1874729916212864/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 2 fl Rpc:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [14274.962976] Lustre: server umount lustre-MDT0001 complete [14288.637352] Lustre: server umount lustre-OST0000 complete [14320.956289] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing set_hostid [14334.320833] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing load_modules_local [14368.579933] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 14051 sec ========= 22:06:58 (1787882818) [14370.649764] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 22:07:00 (1787882820) === [14374.890521] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 22:07:04 (1787882824) === [14420.689664] Lustre: DEBUG MARKER: oleg448-server.virtnet: executing unload_modules_local [14423.685947] Key type lgssc unregistered [14423.992289] LNet: 390679:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [14424.008970] LNetError: 390679:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [14424.029603] LNet: Removed LNI 192.168.204.148@tcp [14425.065366] Key type .llcrypt unregistered [14425.068811] Key type ._llcrypt unregistered