[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 551739039 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524588K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001018] APIC: Switch to symmetric I/O mode setup [ 0.002392] x2apic enabled [ 0.003006] Switched APIC routing to physical x2apic. [ 0.004009] kvm-guest: setup PV IPIs [ 0.006908] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007024] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008011] pid_max: default: 32768 minimum: 301 [ 0.010013] LSM: Security Framework initializing [ 0.011041] Yama: becoming mindful. [ 0.012031] SELinux: Initializing. [ 0.013092] *** VALIDATE selinux *** [ 0.021809] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026567] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.027199] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029208] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030191] *** VALIDATE tmpfs *** [ 0.031500] *** VALIDATE proc *** [ 0.032225] *** VALIDATE cgroup *** [ 0.033007] *** VALIDATE cgroup2 *** [ 0.034280] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.035154] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.036016] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.037035] Spectre V2 : User space: Vulnerable [ 0.038013] Speculative Store Bypass: Vulnerable [ 0.042095] debug: unmapping init [mem 0xffffffffab259000-0xffffffffab260fff] [ 0.045503] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.047202] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.048058] ... version: 2 [ 0.049015] ... bit width: 48 [ 0.050017] ... generic registers: 4 [ 0.051014] ... value mask: 0000ffffffffffff [ 0.052014] ... max period: 00007fffffffffff [ 0.053018] ... fixed-purpose events: 3 [ 0.054027] ... event mask: 000000070000000f [ 0.056428] rcu: Hierarchical SRCU implementation. [ 0.058930] smp: Bringing up secondary CPUs ... [ 0.059790] x86: Booting SMP configuration: [ 0.060027] .... node #0, CPUs: #1 #2 #3 [ 0.067160] smp: Brought up 1 node, 4 CPUs [ 0.069012] smpboot: Max logical packages: 1 [ 0.070010] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.145386] node 0 deferred pages initialised in 72ms [ 0.149087] devtmpfs: initialized [ 0.150669] x86/mm: Memory block size: 128MB [ 0.153516] gcov: version magic: 0x41383552 [ 0.155685] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.163070] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.164399] pinctrl core: initialized pinctrl subsystem [ 0.165338] [ 0.165804] ************************************************************* [ 0.166015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.167013] ** ** [ 0.168016] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.169018] ** ** [ 0.170014] ** This means that this kernel is built to expose internal ** [ 0.171014] ** IOMMU data structures, which may compromise security on ** [ 0.172014] ** your system. ** [ 0.173052] ** ** [ 0.174046] ** If you see this message and you are not debugging the ** [ 0.175012] ** kernel, report this immediately to your vendor! ** [ 0.176014] ** ** [ 0.177014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.178016] ************************************************************* [ 0.179881] NET: Registered protocol family 16 [ 0.180445] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.181068] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.182077] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.184215] cpuidle: using governor menu [ 0.188505] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.189481] PCI: Using configuration type 1 for base access [ 0.190208] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.199030] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.205065] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.208479] cryptd: max_cpu_qlen set to 1000 [ 0.215297] ACPI: Added _OSI(Module Device) [ 0.217013] ACPI: Added _OSI(Processor Device) [ 0.218014] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.219011] ACPI: Added _OSI(Processor Aggregator Device) [ 0.224240] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.233757] ACPI: Interpreter enabled [ 0.235065] ACPI: PM: (supports S0 S3 S4 S5) [ 0.237113] ACPI: Using IOAPIC for interrupt routing [ 0.239307] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.256744] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.353000] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.362050] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.367074] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.374257] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.381389] acpiphp: Slot [2] registered [ 0.383126] acpiphp: Slot [5] registered [ 0.384349] acpiphp: Slot [6] registered [ 0.386265] acpiphp: Slot [7] registered [ 0.388227] acpiphp: Slot [8] registered [ 0.390208] acpiphp: Slot [9] registered [ 0.391316] acpiphp: Slot [10] registered [ 0.393132] acpiphp: Slot [3] registered [ 0.395447] acpiphp: Slot [4] registered [ 0.397114] acpiphp: Slot [11] registered [ 0.400283] acpiphp: Slot [12] registered [ 0.402128] acpiphp: Slot [13] registered [ 0.404431] acpiphp: Slot [14] registered [ 0.408147] acpiphp: Slot [15] registered [ 0.409000] acpiphp: Slot [16] registered [ 0.409000] acpiphp: Slot [17] registered [ 0.409000] acpiphp: Slot [18] registered [ 0.409097] acpiphp: Slot [19] registered [ 0.410000] acpiphp: Slot [20] registered [ 0.412098] acpiphp: Slot [21] registered [ 0.414091] acpiphp: Slot [22] registered [ 0.416142] acpiphp: Slot [23] registered [ 0.418093] acpiphp: Slot [24] registered [ 0.419093] acpiphp: Slot [25] registered [ 0.421150] acpiphp: Slot [26] registered [ 0.422093] acpiphp: Slot [27] registered [ 0.424109] acpiphp: Slot [28] registered [ 0.426118] acpiphp: Slot [29] registered [ 0.427114] acpiphp: Slot [30] registered [ 0.429110] acpiphp: Slot [31] registered [ 0.431060] PCI host bridge to bus 0000:00 [ 0.432019] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.434054] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.436053] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.439028] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.441027] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.443026] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.445196] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.447643] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.451451] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.461015] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.465015] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.467020] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.469015] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.473021] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.477320] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.479976] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.482061] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.485000] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.491018] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.504020] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.510031] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.517973] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.524024] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.532023] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.550018] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.561435] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.568019] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.575020] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.595022] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.609946] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.617025] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.623018] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.642019] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.654198] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.662413] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.671014] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.676000] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.676000] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.676000] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.681018] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.690000] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.695000] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.697045] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.705020] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.722024] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.736121] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.738540] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.741535] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.743545] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.744000] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.749068] iommu: Default domain type: Passthrough [ 0.750000] SCSI subsystem initialized [ 0.750000] ACPI: bus type USB registered [ 0.750000] usbcore: registered new interface driver usbfs [ 0.750000] usbcore: registered new interface driver hub [ 0.750000] usbcore: registered new device driver usb [ 0.750000] pps_core: LinuxPPS API ver. 1 registered [ 0.750000] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.750000] PTP clock support registered [ 0.750000] EDAC MC: Ver: 3.0.0 [ 0.750182] PCI: Using ACPI for IRQ routing [ 0.771162] NetLabel: Initializing [ 0.772014] NetLabel: domain hash size = 128 [ 0.783031] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.786100] NetLabel: unlabeled traffic allowed by default [ 0.789142] vgaarb: loaded [ 0.814230] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.823020] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.849438] clocksource: Switched to clocksource kvm-clock [ 1.120618] VFS: Disk quotas dquot_6.6.0 [ 1.122716] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.164100] *** VALIDATE ramfs *** [ 1.165539] *** VALIDATE hugetlbfs *** [ 1.167528] pnp: PnP ACPI init [ 1.170585] pnp: PnP ACPI: found 6 devices [ 1.188294] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.191974] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.194292] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.196706] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.199452] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.202061] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.205314] NET: Registered protocol family 2 [ 1.208180] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.213658] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.217880] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.223485] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.226919] TCP: Hash tables configured (established 65536 bind 65536) [ 1.229865] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.233018] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.235982] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.239675] NET: Registered protocol family 1 [ 1.242169] RPC: Registered named UNIX socket transport module. [ 1.244293] RPC: Registered udp transport module. [ 1.245866] RPC: Registered tcp transport module. [ 1.247423] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.249834] NET: Registered protocol family 44 [ 1.251780] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.253896] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.255793] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.258106] PCI: CLS 0 bytes, default 64 [ 1.260679] Unpacking initramfs... [ 4.561522] debug: unmapping init [mem 0xffff98eefcc54000-0xffff98eefffbffff] [ 4.583839] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 4.593143] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 4.605709] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 5.482837] Initialise system trusted keyrings [ 5.484754] Key type blacklist registered [ 5.486854] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 5.496548] zbud: loaded [ 5.499850] *** VALIDATE nfs *** [ 5.500981] *** VALIDATE nfs4 *** [ 5.502778] pstore: using deflate compression [ 5.511043] Platform Keyring initialized [ 5.695686] NET: Registered protocol family 38 [ 5.713974] Key type asymmetric registered [ 5.716914] Asymmetric key parser 'x509' registered [ 5.721434] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 5.729605] io scheduler mq-deadline registered [ 5.733813] io scheduler kyber registered [ 5.751941] io scheduler bfq registered [ 5.766703] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 5.788814] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 5.794769] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 5.801807] ACPI: Power Button [PWRF] [ 5.809794] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 5.823300] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 5.838185] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 5.848834] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 5.890477] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 5.919675] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 5.948671] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 5.953745] Non-volatile memory driver v1.3 [ 5.955632] Linux agpgart interface v0.103 [ 5.990653] virtio_blk virtio1: [vda] 146216 512-byte logical blocks (74.9 MB/71.4 MiB) [ 5.993652] vda: detected capacity change from 0 to 74862592 [ 6.244652] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 6.247975] vdb: detected capacity change from 0 to 1073741824 [ 6.285742] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 6.295178] vdc: detected capacity change from 0 to 2621440000 [ 6.352453] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 6.355065] vdd: detected capacity change from 0 to 2621440000 [ 6.382193] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 6.385817] vde: detected capacity change from 0 to 4294967296 [ 6.440592] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 6.450612] vdf: detected capacity change from 0 to 4294967296 [ 6.480388] libphy: Fixed MDIO Bus: probed [ 6.494728] usbcore: registered new interface driver usbserial_generic [ 6.502729] usbserial: USB Serial support registered for generic [ 6.569214] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 6.638600] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 6.641136] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 6.647198] mousedev: PS/2 mouse device common for all mice [ 6.661616] rtc_cmos 00:05: RTC can wake from S4 [ 6.680036] rtc_cmos 00:05: registered as rtc0 [ 6.686395] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 6.695659] intel_pstate: CPU model not supported [ 6.702957] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 6.721711] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 6.727687] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 6.807512] hid: raw HID events driver (C) Jiri Kosina [ 6.810106] usbcore: registered new interface driver usbhid [ 6.812329] usbhid: USB HID core driver [ 6.814563] drop_monitor: Initializing network drop monitor service [ 6.817966] Initializing XFRM netlink socket [ 6.820669] NET: Registered protocol family 10 [ 6.825545] Segment Routing with IPv6 [ 6.827471] NET: Registered protocol family 17 [ 6.830446] mpls_gso: MPLS GSO support [ 6.838760] RAS: Correctable Errors collector initialized. [ 6.840078] AVX version of gcm_enc/dec engaged. [ 6.842534] AES CTR mode by8 optimization enabled [ 7.552783] sched_clock: Marking stable (7552760130, 0)->(9608258953, -2055498823) [ 7.557663] registered taskstats version 1 [ 7.562366] Loading compiled-in X.509 certificates [ 7.565947] zswap: loaded using pool lzo/zbud [ 7.609323] Key type big_key registered [ 7.627264] Key type encrypted registered [ 7.629613] ima: No TPM chip found, activating TPM-bypass! [ 7.633307] ima: Allocated hash algorithm: sha1 [ 7.635845] ima: No architecture policies found [ 7.639843] evm: Initialising EVM extended attributes: [ 7.643333] evm: security.selinux [ 7.644925] evm: security.ima [ 7.646708] evm: security.capability [ 7.648813] evm: HMAC attrs: 0x1 [ 7.653894] rtc_cmos 00:05: setting system clock to 2026-08-24 05:09:15 UTC (1787548155) [ 7.662296] debug: unmapping init [mem 0xffffffffac203000-0xffffffffac3fffff] [ 7.666632] debug: unmapping init [mem 0xffffffffaaf82000-0xffffffffab258fff] [ 7.672092] Write protecting the kernel read-only data: 28672k [ 7.676336] debug: unmapping init [mem 0xffffffffa9603000-0xffffffffa97fffff] [ 7.680772] debug: unmapping init [mem 0xffffffffa9f14000-0xffffffffa9ffffff] [ 7.792457] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 7.817893] systemd[1]: Detected virtualization kvm. [ 7.824239] systemd[1]: Detected architecture x86-64. [ 7.829379] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 7.932632] systemd[1]: No hostname configured. [ 8.045042] systemd[1]: Set hostname to . [ 8.050456] random: systemd: uninitialized urandom read (16 bytes read) [ 8.053505] systemd[1]: Initializing machine ID from random generator. [ 8.597523] random: ln: uninitialized urandom read (6 bytes read) [ 8.791722] random: systemd: uninitialized urandom read (16 bytes read) [ 8.794643] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 8.840527] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 8.853802] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Timers. [ OK ] Reached target Local File Systems. [ OK ] Started Memstrack Anylazing Service. Starting Create Volatile Files and Directories... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Swap. Starting Setup Virtual Console... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. Starting Apply Kernel Variables... [ OK ] Reached target Initrd Root Device. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 11.296247] device-mapper: uevent: version 1.0.3 [ 11.305247] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ 13.771024] virtio_net virtio0 ens2: renamed from eth0 [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 13.918597] random: fast init done [ 14.457964] scsi host0: ata_piix [ 14.532997] scsi host1: ata_piix [ 14.541126] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 14.553554] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 20.164175] random: crng init done [ 20.169090] random: 7 urandom warning(s) missed due to ratelimiting [ 22.175518] dracut-initqueue[584]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 25.090153] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Timers. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Swap. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 28.123915] printk: systemd: 26 output lines suppressed due to ratelimiting [ 29.212646] SELinux: Disabled at runtime. [ 29.375217] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 29.400455] systemd[1]: Detected virtualization kvm. [ 29.414068] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 31.358204] systemd[1]: initrd-switch-root.service: Succeeded. [ 31.368383] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 31.380918] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 31.390164] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 31.396677] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 31.420927] systemd[1]: Starting Journal Service... Starting Journal Service... [ 31.440579] systemd[1]: Started Forward Password Requests to Wall Directory Watch. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Created slice system-getty.slice. Mounting POSIX Message Queue File System... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on initctl Compatibility Named Pipe. Starting Apply Kernel Variables... [ OK ] Reached target rpc_pipefs.target. Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on Process Core Dump Socket. [ OK ] Stopped target Switch Root. Mounting Kernel Debug File System... [ OK ] Created slice system-sshd\x2dkeygen.slice. [ 31.808214] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Stopped target Initrd Root File System. Mounting Huge Pages File System... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. Starting Create list of required st…ce nodes for the current kernel... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Reached target Paths. Starting Remount Root and Kernel File Systems... [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Stopped target Initrd File Systems. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Apply Kernel Variables. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted Huge Pages File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Journal Service. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. Starting Configure read-only root support... Starting Flush Journal to Persistent Storage... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 33.613568] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 35.097643] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 35.364634] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 35.918113] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 36.036535] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (8s / no limit) [** ] A start job is running for Configur…-only root support (9s / no limit)[ 40.963390] Key type dns_resolver registered [*** ] A start job is running for Configur…-only root support (9s / no limit) [ *** ] A start job is running for Configur…only root support (10s / no limit)[ 41.898712] NFS: Registering the id_resolver key type [ 41.900435] Key type id_resolver registered [ 41.903234] Key type id_legacy registered [ *** ] A start job is running for Configur…only root support (10s / no limit) [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Load/Save Random Seed. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started irqbalance daemon. [ OK ] Started daily update of the root trust anchor for DNSSEC. Starting Restore /run/initramfs on shutdown... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started D-Bus System Message Bus. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. Starting Login Service... Starting Network Manager... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... [ OK ] Started Login Service. Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ 52.602142] hrtimer: interrupt took 3085638 ns [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg115-server login: [ 119.615509] libcfs: loading out-of-tree module taints kernel. [ 119.789346] Key type ._llcrypt registered [ 119.793474] Key type .llcrypt registered [ 120.048793] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_hostid [ 147.340634] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 149.078493] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 149.091920] alg: No test for adler32 (adler32-zlib) [ 150.658449] Lustre: Lustre: Build Version: 2.17.57_83_g6b3cfd8 [ 151.671616] LNet: Added LNI 192.168.201.115@tcp [8/256/0/180] [ 153.503171] Key type lgssc registered [ 155.308966] Lustre: Echo OBD driver; http://www.lustre.org/ [ 175.780283] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 222.804576] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 238.294867] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 238.315905] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 239.433590] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 239.444961] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 239.523395] Lustre: lustre-MDT0000: new disk, initializing [ 239.574733] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 239.582419] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 245.028490] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 261.037551] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 261.217990] Lustre: 6517:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 261.261457] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 261.265727] Lustre: Skipped 1 previous similar message [ 261.387743] Lustre: lustre-MDT0001: new disk, initializing [ 261.452933] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 261.482868] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 261.500931] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 267.036786] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 272.490919] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 283.055900] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 283.382987] Lustre: lustre-OST0000: new disk, initializing [ 283.388260] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 283.391413] Lustre: 8456:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 283.445317] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 290.995832] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 291.389530] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 291.408514] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 291.588096] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 307.708039] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 307.872934] Lustre: lustre-OST0001: new disk, initializing [ 307.876831] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 307.881866] Lustre: 9529:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 307.938980] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 315.963311] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 316.955444] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 316.965174] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 317.079883] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 329.295296] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 337.465468] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 349.059115] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing check_logdir /tmp/testlogs/ [ 356.081119] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing yml_node [ 362.263411] Lustre: DEBUG MARKER: Client: 2.17.57.83 [ 365.531313] Lustre: DEBUG MARKER: MDS: 2.17.57.83 [ 368.777447] Lustre: DEBUG MARKER: OSS: 2.17.57.83 [ 371.129523] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Mon Aug 24 01:15:17 EDT 2026 [ 393.185213] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 394.958367] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 414.176447] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 414.177412] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 414.205595] Lustre: Skipped 1 previous similar message [ 414.222495] Lustre: Skipped 3 previous similar messages [ 418.140716] Lustre: server umount lustre-MDT0000 complete [ 422.777527] LustreError: 6511:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787548570 with bad export cookie 8687294387547265376 [ 422.794246] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 422.810505] LustreError: 6511:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 424.416771] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 424.420481] LustreError: 8464:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 424.446469] Lustre: Skipped 2 previous similar messages [ 424.448187] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 424.502785] LustreError: 8464:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 427.425432] LustreError: 6524:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 427.426807] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 427.444430] LustreError: 6524:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 427.451348] Lustre: Skipped 2 previous similar messages [ 432.548978] LustreError: 12173:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 432.550367] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 432.606334] LustreError: 12173:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 432.615342] Lustre: Skipped 1 previous similar message [ 437.215622] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 437.633903] Lustre: server umount lustre-MDT0001 complete [ 442.059350] Lustre: server umount lustre-OST0000 complete [ 446.618992] Lustre: server umount lustre-OST0001 complete [ 454.726861] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_hostid [ 465.236629] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 507.836565] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 508.095866] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 508.120218] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 508.209943] Lustre: lustre-MDT0000: new disk, initializing [ 508.323631] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 508.358227] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 513.750381] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 526.500587] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 526.585634] Lustre: 15088:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 526.626847] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 526.629916] Lustre: Skipped 1 previous similar message [ 526.703332] Lustre: lustre-MDT0001: new disk, initializing [ 526.842869] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 526.904402] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 526.914652] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 532.585848] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 544.695110] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 551.809536] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 559.398651] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 559.679339] Lustre: lustre-OST0000: new disk, initializing [ 559.682909] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 559.690665] Lustre: 17057:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 559.788855] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 561.737512] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 561.741612] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 561.875954] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 568.140680] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 580.663632] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 585.134357] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 585.376573] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 590.928074] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 591.315497] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 597.986976] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 598.001281] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 598.033585] Lustre: Skipped 1 previous similar message [ 598.054565] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 598.062379] Lustre: Skipped 1 previous similar message [ 608.223580] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 608.253387] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 608.264448] Lustre: Skipped 3 previous similar messages [ 608.620373] Lustre: server umount lustre-OST0000 complete [ 613.346656] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 613.379229] Lustre: Skipped 1 previous similar message [ 619.613489] Lustre: server umount lustre-MDT0000 complete [ 623.587154] LustreError: 15100:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 623.605879] LustreError: 15100:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 623.888295] LustreError: 15081:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787548771 with bad export cookie 8687294387547266916 [ 623.891496] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 623.908941] LustreError: 15081:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 624.405370] Lustre: server umount lustre-MDT0001 complete [ 629.261707] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 01:19:36 (1787548776) [ 638.189829] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 638.667675] LustreError: 19325:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 638.722887] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 643.867215] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 648.167271] LustreError: 19324:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 648.188848] LustreError: 19324:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 654.705483] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 659.319879] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 668.606846] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 674.983715] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 682.889115] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 690.111328] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 699.790527] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 704.825855] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 715.238605] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 715.277035] Lustre: Skipped 3 previous similar messages [ 715.279449] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 715.306547] Lustre: Skipped 6 previous similar messages [ 718.242995] Lustre: server umount lustre-MDT0000 complete [ 720.374970] LustreError: 19324:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 720.414158] LustreError: 19324:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 722.281950] LustreError: 19306:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787548870 with bad export cookie 8687294387547267791 [ 722.301324] LustreError: 19306:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 722.302700] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 722.963097] Lustre: server umount lustre-MDT0001 complete [ 736.603634] Lustre: server umount lustre-OST0000 complete [ 763.927852] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 775.649948] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 776.002640] LustreError: 23709:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 776.036142] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 776.039233] Lustre: Skipped 2 previous similar messages [ 779.556776] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 793.738766] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 800.802352] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 804.423401] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 812.990260] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 822.426901] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 833.113132] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 833.284773] Lustre: lustre-OST0001: new disk, initializing [ 833.289507] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 833.297494] Lustre: 26026:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 834.567955] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 834.579474] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 834.767106] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 840.777488] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 852.292280] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 857.392417] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 875.496317] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 875.515931] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 875.520653] Lustre: Skipped 1 previous similar message [ 875.551210] Lustre: Skipped 2 previous similar messages [ 878.756074] Lustre: server umount lustre-MDT0000 complete [ 880.609968] LustreError: 23709:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 880.638582] LustreError: 23709:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 882.920314] LustreError: 23691:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787549030 with bad export cookie 8687294387547268645 [ 882.931600] LustreError: 23691:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 882.935982] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 883.296462] Lustre: server umount lustre-MDT0001 complete [ 898.072717] Lustre: server umount lustre-OST0000 complete [ 902.114519] Lustre: 3650:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787549033/real 1787549033] req@ffff98ee424b3480 x1874380245440000/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787549049 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 902.169412] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 902.188621] Lustre: Skipped 2 previous similar messages [ 903.026977] Lustre: server umount lustre-OST0001 complete [ 920.483646] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 931.707876] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 932.263227] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 932.272740] Lustre: Skipped 3 previous similar messages [ 937.146140] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 947.137385] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 952.952296] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 956.341185] Lustre: 30197:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 966.524248] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 974.348661] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 984.256909] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 991.030262] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 998.931837] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1003.099864] Lustre: 32077:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1010.152990] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1010.171775] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1010.177549] Lustre: Skipped 1 previous similar message [ 1010.208360] Lustre: Skipped 4 previous similar messages [ 1018.539822] Lustre: server umount lustre-MDT0000 complete [ 1020.405441] LustreError: 29044:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1020.429154] LustreError: 29044:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 13 previous similar messages [ 1023.231980] LustreError: 29029:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787549171 with bad export cookie 8687294387547269940 [ 1023.243085] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1023.246107] LustreError: 29029:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1023.530680] Lustre: server umount lustre-MDT0001 complete [ 1038.542184] Lustre: server umount lustre-OST0000 complete [ 1041.439190] Lustre: 3651:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787549173/real 1787549173] req@ffff98ef4d193800 x1874380245494016/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787549189 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1043.067362] Lustre: server umount lustre-OST0001 complete [ 1052.031702] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 01:26:38 (1787549198) [ 1082.542675] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 1096.816682] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1102.528509] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 1115.063909] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1120.806598] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 1125.366336] Lustre: 36034:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1133.173391] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1141.821479] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 1154.245881] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1161.663288] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 1169.483414] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1173.214884] Lustre: 37915:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1180.644395] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1180.649364] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1180.679406] Lustre: Skipped 5 previous similar messages [ 1180.711089] Lustre: Skipped 7 previous similar messages [ 1184.595790] Lustre: server umount lustre-MDT0000 complete [ 1188.264560] LustreError: 36036:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787549336 with bad export cookie 8687294387547271046 [ 1188.271308] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1189.007125] Lustre: server umount lustre-MDT0001 complete [ 1205.154551] Lustre: server umount lustre-OST0000 complete [ 1207.071255] Lustre: 3651:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787549338/real 1787549338] req@ffff98ee43c52d80 x1874380245548032/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787549354 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1209.311292] Lustre: 3650:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787549341/real 1787549341] req@ffff98ee437c3100 x1874380245548288/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787549357 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1210.065384] Lustre: server umount lustre-OST0001 complete [ 1220.951396] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 01:29:27 (1787549367) [ 1238.804857] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 1251.774775] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1252.373111] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1252.384173] Lustre: Skipped 7 previous similar messages [ 1256.549444] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 1265.879146] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1270.130794] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 1273.352225] Lustre: 41204:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1280.184495] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1286.878843] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 1291.943195] LustreError: 42154:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1291.955650] LustreError: 42154:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 18 previous similar messages [ 1295.684690] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1302.253290] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 1310.068126] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1314.377330] Lustre: 43083:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1315.198907] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1315.201101] Lustre: Skipped 1 previous similar message [ 1318.380376] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1318.398849] Lustre: Skipped 4 previous similar messages [ 1324.351986] Lustre: server umount lustre-MDT0000 complete [ 1329.807450] LustreError: 40037:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787549477 with bad export cookie 8687294387547272145 [ 1329.819339] LustreError: 40037:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1329.832128] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1337.012334] Lustre: server umount lustre-MDT0001 complete [ 1347.007907] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1355.179891] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0x788f7837dcfff83f [ 1355.196523] Lustre: MGC192.168.201.115@tcp: Connection restored to 0@lo (at 0@lo) [ 1360.707504] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 1364.915365] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 1370.224397] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1375.938515] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 1376.233419] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1376.244496] Lustre: Skipped 1 previous similar message [ 1384.128047] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1385.870432] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1392.009961] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1393.286067] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1401.312804] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1405.382959] LustreError: 45703:0:(ldlm_resource.c:1207:ldlm_resource_complain()) MGS: namespace resource [0x65727473756c:0x0:0x0].0x0 (ffff98ef53a4a000) refcount nonzero (1) after lock cleanup; forcing cleanup. [ 1405.494455] Lustre: server umount lustre-MDT0000 complete [ 1409.717067] LustreError: 40036:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787549557 with bad export cookie 8687294387547273279 [ 1409.717735] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1409.723058] LustreError: 40036:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 1410.066534] Lustre: server umount lustre-MDT0001 complete [ 1424.837350] Lustre: server umount lustre-OST0000 complete [ 1427.680486] Lustre: 3652:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787549559/real 1787549559] req@ffff98ef7cc21180 x1874380245632640/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787549575 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1430.042858] Lustre: server umount lustre-OST0001 complete [ 1440.569395] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 01:33:06 (1787549586) [ 1460.046958] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 1470.748377] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1476.300295] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 1485.973751] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1490.006816] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 1492.715474] Lustre: 48848:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1499.836600] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1506.311972] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 1517.944270] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1525.049085] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 1533.422623] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1538.165266] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1549.304454] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1549.309272] Lustre: Skipped 19 previous similar messages [ 1551.739489] Lustre: server umount lustre-MDT0000 complete [ 1555.437857] LustreError: 47677:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787549703 with bad export cookie 8687294387547274084 [ 1555.441867] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1555.445706] LustreError: 47677:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1555.763424] Lustre: server umount lustre-MDT0001 complete [ 1569.581608] Lustre: server umount lustre-OST0000 complete [ 1584.673630] Lustre: server umount lustre-OST0001 complete [ 1595.226407] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 01:35:41 (1787549741) [ 1598.568708] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1600.688441] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 01:35:47 (1787549747) [ 1611.859212] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_hostid [ 1620.521839] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 1646.437228] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1646.783403] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1646.900126] Lustre: lustre-MDT0000: new disk, initializing [ 1646.991758] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1651.520714] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 1660.952911] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1667.624334] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1667.872242] Lustre: lustre-OST0000: new disk, initializing [ 1667.877391] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1667.883141] Lustre: Skipped 1 previous similar message [ 1667.891835] Lustre: 54923:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1669.425943] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1669.438190] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1669.480077] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1674.829324] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 1684.709273] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1687.859820] Lustre: DEBUG MARKER: create test files [ 1720.800329] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1720.814279] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1720.833329] Lustre: Skipped 12 previous similar messages [ 1726.248555] Lustre: server umount lustre-OST0000 complete [ 1731.686705] Lustre: server umount lustre-MDT0000 complete [ 1747.187286] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing unload_modules_local [ 1750.249265] Key type lgssc unregistered [ 1750.585774] LNet: 56560:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1750.589379] LNetError: 56560:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1750.616364] LNet: Removed LNI 192.168.201.115@tcp [ 1751.577119] Key type .llcrypt unregistered [ 1751.579617] Key type ._llcrypt unregistered [ 1791.530980] Key type ._llcrypt registered [ 1791.536147] Key type .llcrypt registered [ 1791.642170] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 1793.467874] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1793.522935] alg: No test for adler32 (adler32-zlib) [ 1794.892185] Lustre: Lustre: Build Version: 2.17.57_83_g6b3cfd8 [ 1795.666468] LNet: Added LNI 192.168.201.115@tcp [8/256/0/180] [ 1797.503172] Key type lgssc registered [ 1799.743126] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1808.478467] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1808.492795] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1809.981207] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1817.092730] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 1826.348724] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1834.278373] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1834.575243] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1839.603352] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1839.693475] Lustre: 59299:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1839.698052] Lustre: 59299:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1839.701616] Lustre: 59299:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1839.707406] Lustre: 59299:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1839.711733] Lustre: 59299:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1839.719724] Lustre: 59299:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1841.370719] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 1849.415294] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1855.309599] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1861.150840] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1870.304722] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1870.313054] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1870.326621] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1875.427019] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1876.397898] Lustre: server umount lustre-OST0000 complete [ 1881.999548] Lustre: server umount lustre-MDT0000 complete [ 1893.770889] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing unload_modules_local [ 1896.034495] Key type lgssc unregistered [ 1896.330887] LNet: 60909:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1896.342021] LNetError: 60909:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1896.368360] LNet: Removed LNI 192.168.201.115@tcp [ 1897.292812] Key type .llcrypt unregistered [ 1897.294948] Key type ._llcrypt unregistered [ 1940.445741] Key type ._llcrypt registered [ 1940.449040] Key type .llcrypt registered [ 1940.621462] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 1942.314526] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1942.363930] alg: No test for adler32 (adler32-zlib) [ 1943.445638] Lustre: Lustre: Build Version: 2.17.57_83_g6b3cfd8 [ 1943.678664] LNet: Added LNI 192.168.201.115@tcp [8/256/0/180] [ 1945.311203] Key type lgssc registered [ 1946.163336] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1954.075367] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1954.109659] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1955.593195] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1960.312831] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 1969.267134] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1978.571190] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1979.166677] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1980.208577] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1980.310939] Lustre: 63741:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1980.321758] Lustre: 63741:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1980.331806] Lustre: 63741:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1980.339295] Lustre: 63741:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1980.346814] Lustre: 63741:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1980.356755] Lustre: 63741:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1986.380195] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 1995.612654] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2002.448609] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 2011.109664] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2011.117737] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2011.129834] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2016.229234] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2016.679445] Lustre: server umount lustre-OST0000 complete [ 2021.688786] Lustre: server umount lustre-MDT0000 complete [ 2038.078672] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing unload_modules_local [ 2041.968283] Key type lgssc unregistered [ 2042.409274] LNet: 65305:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2042.423803] LNetError: 65305:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2042.455753] LNet: Removed LNI 192.168.201.115@tcp [ 2043.524340] Key type .llcrypt unregistered [ 2043.527222] Key type ._llcrypt unregistered [ 2068.490529] Key type ._llcrypt registered [ 2068.493687] Key type .llcrypt registered [ 2068.643113] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_hostid [ 2083.949918] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 2085.351988] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2085.475936] alg: No test for adler32 (adler32-zlib) [ 2086.667864] Lustre: Lustre: Build Version: 2.17.57_83_g6b3cfd8 [ 2087.036974] LNet: Added LNI 192.168.201.115@tcp [8/256/0/180] [ 2088.743184] Key type lgssc registered [ 2090.191867] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2129.697623] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2129.721404] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2131.069098] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2131.113951] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2131.246917] Lustre: lustre-MDT0000: new disk, initializing [ 2131.363927] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2131.383315] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2137.064376] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 2149.261635] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2149.365956] Lustre: 68464:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2149.407056] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2149.411490] Lustre: Skipped 1 previous similar message [ 2149.480638] Lustre: lustre-MDT0001: new disk, initializing [ 2149.568033] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2149.607163] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2149.621565] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2154.662289] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 2166.446557] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2172.588654] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2179.974691] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2180.132985] Lustre: lustre-OST0000: new disk, initializing [ 2180.135583] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2180.140377] Lustre: 70433:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2180.226163] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2185.845984] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 2186.272187] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2186.283889] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2186.447909] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2198.120338] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2205.784577] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2206.062559] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2211.031922] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2211.257873] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2217.443985] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2217.444504] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2217.445811] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2217.464339] Lustre: Skipped 1 previous similar message [ 2217.475754] LustreError: Skipped 1 previous similar message [ 2219.888631] Lustre: server umount lustre-OST0000 complete [ 2226.659619] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2226.675179] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2226.723298] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2226.734088] Lustre: Skipped 1 previous similar message [ 2227.692150] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2231.405096] Lustre: server umount lustre-MDT0000 complete [ 2236.499989] LustreError: 68455:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787550384 with bad export cookie 13989262973194343320 [ 2236.511790] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2236.513769] LustreError: 68455:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2237.164823] Lustre: server umount lustre-MDT0001 complete [ 2249.060776] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 01:46:35 (1787550395) [ 2267.077397] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 2322.864858] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_hostid [ 2335.949403] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 2382.209573] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2382.545798] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2382.587731] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2382.687603] Lustre: lustre-MDT0000: new disk, initializing [ 2382.817727] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2382.845186] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2387.363921] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 2400.366607] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2400.567707] Lustre: 76478:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2400.693829] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2400.703273] Lustre: Skipped 1 previous similar message [ 2400.907926] Lustre: lustre-MDT0001: new disk, initializing [ 2401.118548] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2401.223200] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2401.244422] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2407.158495] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 2418.675785] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2425.799169] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2433.290635] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2433.533835] Lustre: lustre-OST0000: new disk, initializing [ 2433.537288] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2433.543165] Lustre: 78450:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2433.622772] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2442.265627] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 2443.948775] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2443.963266] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2444.233936] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2455.302860] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2461.345751] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2461.543354] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2466.597878] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2466.794864] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2469.864894] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2469.866366] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2469.869033] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2469.869042] Lustre: Skipped 2 previous similar messages [ 2469.888800] Lustre: Skipped 1 previous similar message [ 2469.914905] LustreError: Skipped 1 previous similar message [ 2474.984836] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2474.993603] Lustre: Skipped 2 previous similar messages [ 2480.099680] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2480.110025] Lustre: Skipped 1 previous similar message [ 2483.167160] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2483.463117] Lustre: server umount lustre-OST0000 complete [ 2488.289077] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2488.298581] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2488.319050] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2493.473800] Lustre: server umount lustre-MDT0000 complete [ 2495.461280] LustreError: 78977:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2495.491224] LustreError: 78977:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 2497.569522] LustreError: 76470:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787550645 with bad export cookie 13989262973194344202 [ 2497.577801] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2497.814061] Lustre: server umount lustre-MDT0001 complete [ 2509.481227] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 01:50:55 (1787550655) [ 2519.152598] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2520.293184] LustreError: 80866:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2520.581473] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2525.668425] LustreError: 80867:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2526.404841] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 2530.785673] LustreError: 80866:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2535.914158] LustreError: 80867:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2536.412514] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2542.892159] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 2552.708361] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2560.450751] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2570.002458] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2570.260323] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2570.279100] Lustre: Skipped 1 previous similar message [ 2578.377075] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 2588.369546] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2592.776221] Lustre: 83541:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting 0f8664cd-bdcf-44ab-9192-b253d17e6fbf at adminstrative request [ 2596.328897] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2596.349490] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2606.566701] Lustre: MGS: Client 0f8664cd-bdcf-44ab-9192-b253d17e6fbf (at 0@lo) reconnecting [ 2606.583863] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xc223d88e4ac61a7c to 0xc223d88e4ac61d1c [ 2606.599393] Lustre: MGC192.168.201.115@tcp: Connection restored to 0@lo (at 0@lo) [ 2630.360866] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2630.492981] Lustre: lustre-OST0001: new disk, initializing [ 2630.496804] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2630.505580] Lustre: 83934:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2637.706909] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 2637.980033] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2638.011062] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2638.140582] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2649.272401] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2658.791245] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2658.805542] Lustre: Skipped 1 previous similar message [ 2658.819192] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2658.823826] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2658.854459] Lustre: Skipped 3 previous similar messages [ 2660.297531] Lustre: server umount lustre-OST0001 complete [ 2663.923074] LustreError: 82692:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2663.959359] LustreError: 82692:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2668.016305] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2679.263291] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2679.364305] Lustre: server umount lustre-OST0000 complete [ 2684.385690] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2684.394521] Lustre: Skipped 4 previous similar messages [ 2690.053263] Lustre: server umount lustre-MDT0000 complete [ 2694.626122] LustreError: 80863:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2694.686974] LustreError: 80863:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 2696.168817] LustreError: 80846:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787550844 with bad export cookie 13989262973194345756 [ 2696.169493] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2696.185421] LustreError: 80846:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 2696.779416] Lustre: server umount lustre-MDT0001 complete [ 2710.841708] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 01:54:16 (1787550856) [ 2713.118307] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2715.624965] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 01:54:21 (1787550861) [ 2717.640684] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2719.357308] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 01:54:26 (1787550866) [ 2721.377706] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2724.553466] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 01:54:29 (1787550869) [ 2726.168232] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2728.602206] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 01:54:34 (1787550874) [ 2730.577861] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2737.650771] loop: module loaded [ 2737.701755] loop0: detected capacity change from 0 to 4294967296 [ 2737.775731] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2739.923268] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2743.915078] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2748.733491] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2791.886358] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2791.958606] LustreError: 88066:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2791.981942] LustreError: 88066:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2791.991774] LustreError: 88066:0:(tgt_mount.c:2582:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2792.007935] LustreError: 88066:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2815.150368] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_hostid [ 2826.523229] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 2865.871604] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2866.198294] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2866.249894] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2866.358345] Lustre: lustre-MDT0000: new disk, initializing [ 2866.486379] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2866.492867] Lustre: Skipped 1 previous similar message [ 2866.550124] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2871.215297] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 2886.865598] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2887.050131] Lustre: 90935:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2887.346717] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2895.586722] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 2909.371891] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2915.135583] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2922.027168] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2922.331836] Lustre: 92902:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2928.724238] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2928.817246] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2929.137071] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 2941.178595] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2948.914475] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2949.446511] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2957.047547] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2957.487739] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2959.841119] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2959.853825] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2959.861830] LustreError: Skipped 1 previous similar message [ 2959.869583] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2959.869594] Lustre: Skipped 11 previous similar messages [ 2959.916957] Lustre: Skipped 1 previous similar message [ 2966.004650] Lustre: server umount lustre-OST0000 complete [ 2978.474399] Lustre: server umount lustre-MDT0000 complete [ 2980.322152] LustreError: 91743:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2980.335337] LustreError: 91743:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2983.117569] LustreError: 90926:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787551130 with bad export cookie 13989262973194346372 [ 2983.121407] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2983.134307] LustreError: 90926:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2983.912688] Lustre: server umount lustre-MDT0001 complete [ 2994.341663] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 01:59:00 (1787551140) [ 3004.152447] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3005.365224] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 3005.370092] Lustre: Skipped 2 previous similar messages [ 3011.896957] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 3020.816839] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3026.013779] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 3033.267271] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3041.031657] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3050.777225] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3060.715426] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 3074.519256] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3080.249842] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3080.445746] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3085.086761] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3085.298675] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3093.664802] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3093.993793] Lustre: lustre-OST0001: new disk, initializing [ 3093.996806] Lustre: Skipped 2 previous similar messages [ 3094.010345] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3094.015638] Lustre: Skipped 3 previous similar messages [ 3094.020966] Lustre: 98690:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3099.843965] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3099.861329] Lustre: Skipped 2 previous similar messages [ 3099.873481] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3100.128640] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3103.243959] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 3121.301996] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3129.357581] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 3129.859408] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3136.801393] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 3137.004789] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3161.704580] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 3187.801700] Lustre: Failing over lustre-MDT0000 [ 3188.182508] Lustre: server umount lustre-MDT0000 complete [ 3190.245977] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3190.252278] LustreError: Skipped 1 previous similar message [ 3190.256890] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3190.268686] Lustre: Skipped 2 previous similar messages [ 3190.272802] LustreError: 95318:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3190.293366] LustreError: 95318:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 3208.674879] Lustre: 66684:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787551340/real 1787551340] req@00000000bfed0f26 x1874382274472704/t0(0) o400->MGC192.168.201.115@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787551356 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3208.707405] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3217.564568] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 3217.569239] LDISKFS-fs (dm-0): recovery complete [ 3217.586670] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3218.911857] LustreError: 101524:0:(import.c:339:ptlrpc_invalidate_import()) MGS: timeout waiting for callback (1 != 0) [ 3218.921389] LustreError: 101524:0:(import.c:363:ptlrpc_invalidate_import()) @@@ still on sending list req@000000005ace828f x1874382274477952/t0(0) o250->MGC192.168.201.115@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 1787551366 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3218.944237] LustreError: 101524:0:(import.c:373:ptlrpc_invalidate_import()) MGS: Unregistering RPCs found (0). Network is sluggish? Waiting for them to error out. [ 3220.548458] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 3221.390945] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 3221.401131] Lustre: 95313:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3225.569922] Lustre: 95318:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3225.575092] Lustre: 95318:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 3225.591375] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3225.684353] LustreError: 101556:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3225.966799] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 3226.023065] LustreError: 101556:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3226.356393] LustreError: 101556:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3226.376051] LustreError: 101556:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 3226.701122] LustreError: 101556:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3226.719642] LustreError: 101556:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 3226.739230] Lustre: 101556:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3226.754880] Lustre: 101556:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 3227.423990] LustreError: 101556:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3227.429778] LustreError: 101556:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 3227.754223] LustreError: 101556:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3227.764412] LustreError: 101556:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 3228.736286] Lustre: 101556:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3228.751037] Lustre: 101556:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 5 previous similar messages [ 3229.731839] LustreError: 101556:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3229.749489] LustreError: 101556:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 3230.071207] LustreError: 101556:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3230.077479] LustreError: 101556:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 3232.984358] Lustre: 101556:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3233.001605] Lustre: 101556:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 3233.960126] LustreError: 101556:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3233.976102] LustreError: 101556:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 3234.295235] LustreError: 101556:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3234.298550] LustreError: 101556:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 3241.073086] Lustre: 101556:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3241.084772] Lustre: 101556:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 24 previous similar messages [ 3242.053084] LustreError: 101556:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3242.060886] LustreError: 101556:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 3242.383174] LustreError: 101556:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3242.404195] LustreError: 101556:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 3257.345375] Lustre: 101556:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3257.365427] Lustre: 101556:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 50 previous similar messages [ 3258.331476] LustreError: 101556:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3258.387493] LustreError: 101556:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 3258.711133] LustreError: 101556:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3258.715510] LustreError: 101556:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 3281.231921] Lustre: 101556:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3281.249069] Lustre: 101556:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3281.268425] Lustre: 101556:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 893051e1-3778-40c5-8d47-50225c688f51@192.168.201.15@tcp [ 3281.289622] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3281.317578] Lustre: 101556:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@000000002d537cd8 x1874382258507904/t0(8589934769) o101->893051e1-3778-40c5-8d47-50225c688f51@192.168.201.15@tcp:604/0 lens 592/0 e 0 to 0 dl 1787551439 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3281.384744] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3281.415885] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3281.425414] Lustre: lustre-MDT0000: Denying connection for new client 893051e1-3778-40c5-8d47-50225c688f51 (at 192.168.201.15@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3281.503685] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3281.525251] Lustre: Skipped 2 previous similar messages [ 3281.532640] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3281.679827] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:77 to 0x280000401:577) [ 3281.685368] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:45 to 0x2c0000401:545) [ 3301.859396] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3301.876800] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3301.893038] Lustre: Skipped 6 previous similar messages [ 3305.538401] Lustre: server umount lustre-OST0000 complete [ 3322.850133] LustreError: 97142:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3322.858509] LustreError: 97142:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 51 previous similar messages [ 3324.383092] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3324.657393] Lustre: server umount lustre-OST0001 complete [ 3333.089677] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3333.105521] Lustre: Skipped 8 previous similar messages [ 3335.799410] Lustre: server umount lustre-MDT0000 complete [ 3339.016696] LustreError: 97163:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787551486 with bad export cookie 13989262973194397584 [ 3339.025085] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3339.030923] LustreError: 97163:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3339.382783] Lustre: server umount lustre-MDT0001 complete [ 3351.704693] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 02:04:57 (1787551497) [ 3361.116700] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3361.133482] Lustre: *** cfs_fail_loc=197, val=0*** [ 3377.055650] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3382.175501] LustreError: 103535:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.115@tcp: failed processing log, type 4: rc = -110 [ 3387.312328] Lustre: *** cfs_fail_loc=197, val=0*** [ 3387.315436] Lustre: Skipped 65 previous similar messages [ 3407.905331] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3407.910242] Lustre: Skipped 4 previous similar messages [ 3415.152575] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 3423.838611] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3426.611255] Lustre: server umount lustre-OST0000 complete [ 3438.725966] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 02:06:24 (1787551584) [ 3462.458960] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3469.572914] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 3482.462635] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3487.152057] LustreError: 105192:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3487.157316] LustreError: 105192:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3487.166951] LustreError: 105192:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 24, retries 0, failed: rc = -5 [ 3487.488756] Lustre: server umount lustre-MDT0000 complete [ 3534.345157] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_hostid [ 3547.667928] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 3605.666569] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3606.153394] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3606.191898] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3606.273734] Lustre: lustre-MDT0000: new disk, initializing [ 3606.435103] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3612.027576] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 3626.648663] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3626.848492] Lustre: 109371:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3627.243975] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3633.288830] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 3645.241380] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3653.888493] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3662.625869] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3663.104096] Lustre: 111342:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3669.859244] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3671.473431] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 3686.536521] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3695.424782] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3695.749355] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3701.772761] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3702.034748] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3705.827129] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3705.830574] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3705.860723] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3705.891925] LustreError: Skipped 1 previous similar message [ 3705.955678] Lustre: Skipped 11 previous similar messages [ 3710.873096] Lustre: server umount lustre-OST0000 complete [ 3722.426665] Lustre: server umount lustre-MDT0000 complete [ 3725.282278] LustreError: 109384:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3725.293839] LustreError: 109384:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [ 3726.943250] LustreError: 109365:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787551874 with bad export cookie 13989262973194408161 [ 3726.968884] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3727.289062] Lustre: server umount lustre-MDT0001 complete [ 3738.878411] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 02:11:25 (1787551885) [ 3750.047475] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing unload_modules_local [ 3754.077909] Key type lgssc unregistered [ 3754.646186] LNet: 113834:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3754.662342] LNetError: 113834:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3755.694371] LNet: Removed LNI 192.168.201.115@tcp [ 3756.672512] Key type .llcrypt unregistered [ 3756.676268] Key type ._llcrypt unregistered [ 3778.913885] Key type ._llcrypt registered [ 3778.915647] Key type .llcrypt registered [ 3779.130377] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_hostid [ 3797.280964] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 3798.394925] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3798.534906] alg: No test for adler32 (adler32-zlib) [ 3799.578351] Lustre: Lustre: Build Version: 2.17.57_83_g6b3cfd8 [ 3799.856978] LNet: Added LNI 192.168.201.115@tcp [8/256/0/180] [ 3801.607197] Key type lgssc registered [ 3802.883864] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3861.418282] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3861.438329] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3862.769379] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3862.840709] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3862.940726] Lustre: lustre-MDT0000: new disk, initializing [ 3863.017411] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3863.033031] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3869.780924] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 3881.652338] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3882.140807] Lustre: lustre-OST0000: new disk, initializing [ 3882.145623] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3882.153399] Lustre: Skipped 1 previous similar message [ 3882.158869] Lustre: 118012:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3882.247857] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3887.727256] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3887.732671] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3887.771152] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3888.853661] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 3900.714583] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3900.957982] Lustre: lustre-OST0001: new disk, initializing [ 3900.968971] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3900.978161] Lustre: 118877:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3901.102767] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3908.152951] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 3909.767298] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3909.770364] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3909.813577] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3920.353956] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3920.377577] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3920.396506] Lustre: Skipped 1 previous similar message [ 3923.786709] Lustre: server umount lustre-MDT0000 complete [ 3933.369515] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3939.212655] LustreError: 118876:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787552087 with bad export cookie 17650642332100621499 [ 3939.222196] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3939.231176] LustreError: 118876:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3939.307766] Lustre: server umount lustre-OST0000 complete [ 3944.597101] Lustre: server umount lustre-OST0001 complete [ 3955.473588] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 02:15:01 (1787552101) [ 3986.738597] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 02:15:33 (1787552133) [ 3999.167416] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 4027.323838] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 02:16:13 (1787552173) [ 4073.015610] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_hostid [ 4085.071695] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 4128.238225] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4128.586661] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4128.629331] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4128.703972] Lustre: lustre-MDT0000: new disk, initializing [ 4128.804356] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4128.828665] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4134.112529] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 4147.445916] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4147.644685] Lustre: 126634:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4147.747193] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4147.751357] Lustre: Skipped 1 previous similar message [ 4147.868427] Lustre: lustre-MDT0001: new disk, initializing [ 4147.988665] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4148.047215] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4148.062459] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4154.274857] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 4165.813455] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4172.390858] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4180.935144] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4181.292877] Lustre: lustre-OST0000: new disk, initializing [ 4181.301660] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4181.315898] Lustre: 128605:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4181.504738] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4188.934798] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 4190.777560] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4190.788525] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4190.910457] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4199.348247] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4205.936148] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4206.235911] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4211.694873] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4211.926916] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4216.301079] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4216.307354] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4216.333022] Lustre: Skipped 2 previous similar messages [ 4216.345513] Lustre: Skipped 1 previous similar message [ 4221.422362] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4221.427851] Lustre: Skipped 1 previous similar message [ 4226.532599] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4226.535913] Lustre: Skipped 1 previous similar message [ 4228.063189] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4228.376693] Lustre: server umount lustre-OST0000 complete [ 4235.232116] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4235.235147] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4235.273575] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4240.081294] Lustre: server umount lustre-MDT0000 complete [ 4244.302097] LustreError: 126628:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787552392 with bad export cookie 17650642332100622717 [ 4244.311203] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4244.316251] LustreError: 126628:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4244.773143] Lustre: server umount lustre-MDT0001 complete [ 4255.806376] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 02:20:01 (1787552401) [ 4264.257319] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4265.413540] LustreError: 131018:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4265.450036] LustreError: 131018:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 4265.655236] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4271.074171] LustreError: 131019:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4272.450751] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 4275.175624] LustreError: 131018:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4280.297179] LustreError: 131019:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4283.270812] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4287.963202] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 4296.883172] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4304.390774] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4313.101237] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4313.501973] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4313.511960] Lustre: Skipped 1 previous similar message [ 4321.147482] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 4328.999271] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4346.527505] Lustre: *** cfs_fail_loc=159, val=0*** [ 4346.542774] Lustre: Skipped 1 previous similar message [ 4362.192055] Lustre: lustre-MDT0000: Client e7211583-be34-4aa2-8e50-72fd52613770 (at 192.168.201.15@tcp) reconnecting [ 4364.350094] Lustre: *** cfs_fail_loc=159, val=0*** [ 4364.351539] Lustre: Skipped 4 previous similar messages [ 4380.604424] Lustre: lustre-MDT0000: Client e7211583-be34-4aa2-8e50-72fd52613770 (at 192.168.201.15@tcp) reconnecting [ 4400.613103] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4400.618260] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4400.628882] Lustre: Skipped 1 previous similar message [ 4400.639656] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4400.643781] Lustre: Skipped 2 previous similar messages [ 4401.501849] Lustre: server umount lustre-OST0000 complete [ 4406.754958] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4406.765209] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4417.190089] Lustre: server umount lustre-MDT0000 complete [ 4418.016230] LustreError: 131019:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4418.023538] LustreError: 131019:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4421.047848] LustreError: 130998:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787552568 with bad export cookie 17650642332100623606 [ 4421.051607] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4421.073885] LustreError: 130998:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4421.763935] Lustre: server umount lustre-MDT0001 complete [ 4434.817249] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing unload_modules_local [ 4438.284532] Key type lgssc unregistered [ 4438.682276] LNet: 134827:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4438.686784] LNetError: 134827:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4438.708526] LNet: Removed LNI 192.168.201.115@tcp [ 4440.085337] Key type .llcrypt unregistered [ 4440.096705] Key type ._llcrypt unregistered [ 4468.055297] Key type ._llcrypt registered [ 4468.056923] Key type .llcrypt registered [ 4468.117400] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 4469.103575] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4469.289593] alg: No test for adler32 (adler32-zlib) [ 4470.321525] Lustre: Lustre: Build Version: 2.17.57_83_g6b3cfd8 [ 4470.551955] LNet: Added LNI 192.168.201.115@tcp [8/256/0/180] [ 4472.271272] Key type lgssc registered [ 4473.855878] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4482.786586] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4482.829714] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4484.349556] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4490.077458] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 4494.831785] LustreError: 136245:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4494.848378] LustreError: 136245:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4499.370478] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4499.797248] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4505.005965] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 4512.839219] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4518.798895] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4525.729761] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4526.147746] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4530.232684] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4532.202857] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 4540.306175] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4553.466239] Lustre: *** cfs_fail_loc=159, val=0*** [ 4569.007212] Lustre: lustre-MDT0000: Client 55a778d6-112c-4554-bdb1-2bc18dcb042e (at 192.168.201.15@tcp) reconnecting [ 4572.846789] Lustre: *** cfs_fail_loc=159, val=0*** [ 4588.445325] Lustre: lustre-MDT0001: Client 55a778d6-112c-4554-bdb1-2bc18dcb042e (at 192.168.201.15@tcp) reconnecting [ 4590.802331] Lustre: *** cfs_fail_loc=159, val=0*** [ 4590.807525] Lustre: Skipped 5 previous similar messages [ 4606.407779] Lustre: lustre-MDT0001: Client 55a778d6-112c-4554-bdb1-2bc18dcb042e (at 192.168.201.15@tcp) reconnecting [ 4612.608392] Lustre: 139284:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4616.538804] Lustre: *** cfs_fail_loc=159, val=0*** [ 4616.540761] Lustre: Skipped 2 previous similar messages [ 4632.001704] Lustre: lustre-MDT0000: Client 6c4d56d0-15bd-44b4-827b-22bcc6278232 (at 192.168.201.15@tcp) reconnecting [ 4634.547721] Lustre: *** cfs_fail_loc=159, val=0*** [ 4634.562961] Lustre: Skipped 14 previous similar messages [ 4650.413152] Lustre: lustre-MDT0000: Client 6c4d56d0-15bd-44b4-827b-22bcc6278232 (at 192.168.201.15@tcp) reconnecting [ 4651.783909] Lustre: 139571:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4658.161116] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4658.171047] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4658.176423] LustreError: Skipped 2 previous similar messages [ 4658.188259] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4662.584897] Lustre: server umount lustre-OST0000 complete [ 4668.396604] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4668.410146] Lustre: Skipped 1 previous similar message [ 4668.415789] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4668.421475] Lustre: Skipped 2 previous similar messages [ 4672.829744] Lustre: server umount lustre-MDT0000 complete [ 4673.517978] LustreError: 136245:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4673.531486] LustreError: 136245:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4677.246378] LustreError: 139402:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787552825 with bad export cookie 18069033209650170032 [ 4677.246954] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4677.253808] LustreError: 139402:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4677.520305] Lustre: server umount lustre-MDT0001 complete [ 4688.259389] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing unload_modules_local [ 4691.408376] Key type lgssc unregistered [ 4691.834881] LNet: 140629:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4691.843823] LNetError: 140629:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4691.859595] LNet: Removed LNI 192.168.201.115@tcp [ 4692.836631] Key type .llcrypt unregistered [ 4692.844343] Key type ._llcrypt unregistered [ 4728.816150] Key type ._llcrypt registered [ 4728.820974] Key type .llcrypt registered [ 4729.014456] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 4730.899331] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4730.917213] alg: No test for adler32 (adler32-zlib) [ 4732.036800] Lustre: Lustre: Build Version: 2.17.57_83_g6b3cfd8 [ 4732.206480] LNet: Added LNI 192.168.201.115@tcp [8/256/0/180] [ 4733.911604] Key type lgssc registered [ 4735.789849] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4747.922965] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4747.977044] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4749.769607] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4754.685846] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 4760.040631] LustreError: 142048:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4760.067706] LustreError: 142048:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4765.167915] LustreError: 142047:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4767.019608] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4767.678090] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4772.893423] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 4783.893951] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4790.419449] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4799.417065] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4799.895574] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4806.851823] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 4813.214542] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4816.463801] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4819.337206] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4827.733899] Lustre: 144633:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4831.056239] Lustre: 144682:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4832.858764] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4833.650412] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4837.992835] Lustre: 144829:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4838.996510] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4840.645108] Lustre: 144938:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4849.120753] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4849.127309] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4849.140350] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4850.149871] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4850.151087] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4850.159847] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4850.206421] Lustre: Skipped 1 previous similar message [ 4853.286443] Lustre: server umount lustre-OST0000 complete [ 4859.363688] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4859.380731] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4859.403099] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4864.865829] Lustre: server umount lustre-MDT0000 complete [ 4865.510711] LustreError: 142048:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4865.534957] LustreError: 142048:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4869.505800] LustreError: 142030:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787553017 with bad export cookie 6312302929658195097 [ 4869.511778] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4869.524802] LustreError: 142030:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4870.089729] Lustre: server umount lustre-MDT0001 complete [ 4884.614983] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing unload_modules_local [ 4888.236834] Key type lgssc unregistered [ 4888.549437] LNet: 145991:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4888.559885] LNetError: 145991:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4888.576180] LNet: Removed LNI 192.168.201.115@tcp [ 4889.551909] Key type .llcrypt unregistered [ 4889.558174] Key type ._llcrypt unregistered [ 4916.771076] Key type ._llcrypt registered [ 4916.773343] Key type .llcrypt registered [ 4916.880970] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 4918.056666] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4918.317199] alg: No test for adler32 (adler32-zlib) [ 4919.457445] Lustre: Lustre: Build Version: 2.17.57_83_g6b3cfd8 [ 4919.797070] LNet: Added LNI 192.168.201.115@tcp [8/256/0/180] [ 4921.535347] Key type lgssc registered [ 4923.062794] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4932.364372] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4932.390432] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4934.093034] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4939.598788] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 4943.333476] LustreError: 147409:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4943.356367] LustreError: 147409:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4948.450287] LustreError: 147410:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4949.373728] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4949.747893] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4954.883588] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 4963.968308] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4970.454617] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4977.909314] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4978.459785] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4979.510169] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4985.302921] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 4994.842384] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4994.896374] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 5003.032518] Lustre: *** cfs_fail_loc=159, val=0*** [ 5019.591420] Lustre: lustre-MDT0000: Client fdaf1b5b-76d9-4972-b777-ae4ecf5d07db (at 192.168.201.15@tcp) reconnecting [ 5036.015836] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5036.030173] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5036.035994] Lustre: Skipped 1 previous similar message [ 5038.011613] Lustre: server umount lustre-OST0000 complete [ 5042.145065] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5042.150392] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5042.157518] Lustre: Skipped 1 previous similar message [ 5042.160847] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5046.241153] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5046.244596] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5046.271453] Lustre: Skipped 1 previous similar message [ 5048.294457] Lustre: server umount lustre-MDT0000 complete [ 5051.364223] LustreError: 148141:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5051.387720] LustreError: 148141:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 5052.264855] LustreError: 147389:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787553200 with bad export cookie 2446013967258956773 [ 5052.270041] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5052.272918] LustreError: 147389:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5052.611413] Lustre: server umount lustre-MDT0001 complete [ 5067.890193] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing unload_modules_local [ 5071.635257] Key type lgssc unregistered [ 5072.144285] LNet: 151077:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5072.160817] LNetError: 151077:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5072.202492] LNet: Removed LNI 192.168.201.115@tcp [ 5073.867478] Key type .llcrypt unregistered [ 5073.872207] Key type ._llcrypt unregistered [ 5102.710723] Key type ._llcrypt registered [ 5102.711987] Key type .llcrypt registered [ 5102.756982] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 5103.968860] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5104.076143] alg: No test for adler32 (adler32-zlib) [ 5105.085463] Lustre: Lustre: Build Version: 2.17.57_83_g6b3cfd8 [ 5105.390973] LNet: Added LNI 192.168.201.115@tcp [8/256/0/180] [ 5107.095465] Key type lgssc registered [ 5107.981080] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5117.186336] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5117.226874] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5119.038978] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5125.552252] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 5128.169465] LustreError: 152492:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5128.197144] LustreError: 152492:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5133.303413] LustreError: 152493:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5137.861289] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5138.420744] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 5138.746995] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5144.080732] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 5154.160747] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5161.478574] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5171.343668] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5171.991639] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5181.078710] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 5183.287993] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 5188.429481] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 5190.404843] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5213.152518] Lustre: lustre-OST0000: evicting ae1c15b4-7bce-4b66-bd0a-dad801c280af (at 192.168.201.15@tcp) by administrative request [ 5213.163719] Lustre: Skipped 1 previous similar message [ 5244.898636] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5244.901526] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5244.904046] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5244.912064] LustreError: Skipped 1 previous similar message [ 5244.961263] Lustre: Skipped 1 previous similar message [ 5250.029043] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5250.033966] Lustre: Skipped 2 previous similar messages [ 5255.140052] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5255.151482] Lustre: Skipped 1 previous similar message [ 5257.183120] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5257.466758] Lustre: server umount lustre-OST0000 complete [ 5265.387715] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5265.393497] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5265.404469] Lustre: Skipped 1 previous similar message [ 5265.423859] Lustre: Skipped 1 previous similar message [ 5268.984914] Lustre: server umount lustre-MDT0000 complete [ 5270.499083] LustreError: 152492:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5270.512501] LustreError: 152492:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 5273.611477] LustreError: 152475:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787553421 with bad export cookie 9929956741942553856 [ 5273.618434] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5273.627535] LustreError: 152475:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5274.033416] Lustre: server umount lustre-MDT0001 complete [ 5286.301834] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing unload_modules_local [ 5289.438303] Key type lgssc unregistered [ 5289.728883] LNet: 156574:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5289.733781] LNetError: 156574:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5289.746408] LNet: Removed LNI 192.168.201.115@tcp [ 5290.800912] Key type .llcrypt unregistered [ 5290.802510] Key type ._llcrypt unregistered [ 5374.289311] Key type ._llcrypt registered [ 5374.293246] Key type .llcrypt registered [ 5374.449143] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 5376.204512] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5376.243474] alg: No test for adler32 (adler32-zlib) [ 5377.279837] Lustre: Lustre: Build Version: 2.17.57_83_g6b3cfd8 [ 5377.502570] LNet: Added LNI 192.168.201.115@tcp [8/256/0/180] [ 5379.167288] Key type lgssc registered [ 5380.325283] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5389.092433] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5389.109911] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5390.703402] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5396.406159] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 5401.061650] LustreError: 159063:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5401.099683] LustreError: 159063:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5406.192100] LustreError: 159064:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5406.480811] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5406.793628] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5412.697458] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 5421.731163] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5427.397820] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5435.041524] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5435.460623] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5440.534177] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 5443.576897] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 5451.845404] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5453.149517] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5466.082392] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5466.098509] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5466.117109] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5467.125750] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5467.130091] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5467.138376] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5467.170481] Lustre: Skipped 1 previous similar message [ 5470.216636] Lustre: server umount lustre-OST0000 complete [ 5477.345142] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5477.356802] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5481.301787] Lustre: server umount lustre-MDT0000 complete [ 5482.469945] LustreError: 159063:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5482.496612] LustreError: 159063:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 5486.260764] LustreError: 159045:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787553634 with bad export cookie 14517096793868931721 [ 5486.267794] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5486.272021] LustreError: 159045:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5486.660145] Lustre: server umount lustre-MDT0001 complete [ 5501.327827] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing unload_modules_local [ 5505.943573] Key type lgssc unregistered [ 5506.303812] LNet: 162620:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5506.311713] LNetError: 162620:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5507.368231] LNet: Removed LNI 192.168.201.115@tcp [ 5508.572384] Key type .llcrypt unregistered [ 5508.580723] Key type ._llcrypt unregistered [ 5536.660035] Key type ._llcrypt registered [ 5536.663238] Key type .llcrypt registered [ 5536.782912] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 5538.152218] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5538.197795] alg: No test for adler32 (adler32-zlib) [ 5539.236925] Lustre: Lustre: Build Version: 2.17.57_83_g6b3cfd8 [ 5539.482114] LNet: Added LNI 192.168.201.115@tcp [8/256/0/180] [ 5541.231265] Key type lgssc registered [ 5543.029291] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5562.776520] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 02:41:48 (1787553708) [ 5583.605368] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_hostid [ 5595.188536] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 5645.541399] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5645.590682] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5647.083511] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5647.121872] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5647.255647] Lustre: lustre-MDT0000: new disk, initializing [ 5647.390293] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5647.424648] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5653.844479] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 5668.517922] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5668.597259] Lustre: 166887:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5668.624947] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5668.627093] Lustre: Skipped 1 previous similar message [ 5668.670509] Lustre: lustre-MDT0001: new disk, initializing [ 5668.720637] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5668.738973] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5668.743801] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5673.276992] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 5684.171663] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5690.104893] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5696.739314] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5697.268807] Lustre: lustre-OST0000: new disk, initializing [ 5697.274669] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5697.289250] Lustre: 168859:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5697.463406] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5704.394786] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5704.411982] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5704.714703] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5708.048159] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 5721.853318] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5735.409400] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5735.419510] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5735.436827] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5735.451542] Lustre: Skipped 1 previous similar message [ 5740.517951] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5740.539899] Lustre: Skipped 2 previous similar messages [ 5745.649862] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5749.215139] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5750.559276] Lustre: server umount lustre-OST0000 complete [ 5755.874585] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5755.891794] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5755.900547] Lustre: Skipped 2 previous similar messages [ 5760.993092] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5761.002514] Lustre: Skipped 1 previous similar message [ 5762.004483] Lustre: server umount lustre-MDT0000 complete [ 5766.453362] LustreError: 166882:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787553914 with bad export cookie 2301876620969801144 [ 5766.459295] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5766.472576] LustreError: 166882:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5767.101443] Lustre: server umount lustre-MDT0001 complete [ 5779.808128] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing unload_modules_local [ 5783.648066] Key type lgssc unregistered [ 5784.174357] LNet: 170771:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5784.184918] LNetError: 170771:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5784.205654] LNet: Removed LNI 192.168.201.115@tcp [ 5785.312505] Key type .llcrypt unregistered [ 5785.314568] Key type ._llcrypt unregistered [ 5812.640213] Key type ._llcrypt registered [ 5812.642861] Key type .llcrypt registered [ 5812.823837] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 5814.067892] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5814.140236] alg: No test for adler32 (adler32-zlib) [ 5815.545784] Lustre: Lustre: Build Version: 2.17.57_83_g6b3cfd8 [ 5815.848068] LNet: Added LNI 192.168.201.115@tcp [8/256/0/180] [ 5817.599193] Key type lgssc registered [ 5818.930198] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5827.333653] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5827.351403] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5828.946844] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5833.655114] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 5838.306554] LustreError: 172200:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5838.321040] LustreError: 172200:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5843.438958] LustreError: 172201:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5843.476847] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5844.034157] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5849.320957] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 5858.553809] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5867.138734] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5876.044649] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5876.602917] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5884.705354] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 5893.789131] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5900.169589] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5906.399137] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5906.414069] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5906.427911] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5906.438814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.459456] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5907.554959] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5907.599865] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5907.620180] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5907.633797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.690233] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5909.023840] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5909.072220] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5909.092873] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5909.096692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.112149] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5910.108956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.959765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.930914] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5911.935740] Lustre: Skipped 2 previous similar messages [ 5911.954240] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5911.960086] Lustre: Skipped 2 previous similar messages [ 5911.962971] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5911.965061] Lustre: Skipped 2 previous similar messages [ 5911.967864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.976888] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5911.982719] Lustre: Skipped 2 previous similar messages [ 5913.117540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.953974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.177243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.080237] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5916.083871] Lustre: Skipped 3 previous similar messages [ 5916.095455] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5916.100926] Lustre: Skipped 3 previous similar messages [ 5916.108704] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5916.117843] Lustre: Skipped 3 previous similar messages [ 5916.130678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.140422] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5916.146458] Lustre: Skipped 3 previous similar messages [ 5917.128176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.148444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.015140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.014686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.100497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.236877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.286126] LustreError: 172220:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5923.315831] LustreError: 172220:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5923.339501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.511390] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5924.518527] Lustre: Skipped 7 previous similar messages [ 5924.545664] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5924.560920] Lustre: Skipped 7 previous similar messages [ 5924.564687] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5924.569789] Lustre: Skipped 7 previous similar messages [ 5924.573782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.584721] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5924.591953] Lustre: Skipped 7 previous similar messages [ 5925.534741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.718597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.968940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.995913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.088136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.179436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.285334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.400691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.653258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.027825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.960769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.200725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.865347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.967288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.906945] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5940.910925] Lustre: Skipped 14 previous similar messages [ 5940.939327] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5940.952993] Lustre: Skipped 14 previous similar messages [ 5940.959504] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5940.966276] Lustre: Skipped 14 previous similar messages [ 5940.969671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.980873] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5940.986876] Lustre: Skipped 14 previous similar messages [ 5941.932718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.372466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.822260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.461608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.500754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.913045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.147335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.864695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.708763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.342754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.633994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.758504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.374482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.024584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.486864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.679908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.965946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.552877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.799448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.218869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.460377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.528200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.409315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.378390] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5973.383741] Lustre: Skipped 23 previous similar messages [ 5973.405275] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5973.414114] Lustre: Skipped 23 previous similar messages [ 5973.416801] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5973.419478] Lustre: Skipped 23 previous similar messages [ 5973.432894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.445754] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5973.454189] Lustre: Skipped 23 previous similar messages [ 5974.520756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.622107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.408958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.359153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.185808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.022938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.771457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.678403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.064619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.173942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.326819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.350821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.791411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.796191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.174516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.510531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.469576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.687863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.642874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.778647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.846820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.884372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.884812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.967688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.202104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.631806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.868130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.123543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.353162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.769335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.605617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.805228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.766666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.192996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.222277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.205890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.247679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.463577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.231872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.354349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.371939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.548070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.221710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.218909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.798747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.035396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.305845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.179467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.320096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.332667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.631402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.005893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.407571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.708931] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6037.720630] Lustre: Skipped 53 previous similar messages [ 6037.741500] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6037.759562] Lustre: Skipped 53 previous similar messages [ 6037.763454] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6037.769262] Lustre: Skipped 53 previous similar messages [ 6037.772943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.796843] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6037.815078] Lustre: Skipped 53 previous similar messages [ 6038.867966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.782247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.435535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.658456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.819368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.097081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.635451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.089929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.428532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.860994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.046977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.388415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.761705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.121150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.357597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.547053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.078489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.273782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.317996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.518974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.621247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.029961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.258032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.545058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.797929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.238071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.440755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.730577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.867583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.992413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.238466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.646916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.882853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.884272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.947357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.184637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.372941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.657820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.656139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.917951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.269501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.625437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.559561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.611745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.813863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.098585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.459109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.700406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.926615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.662973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.275772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.524091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.505313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.951779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.399217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.679140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.037702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.399164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.672817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.755029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.912757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.086593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.549707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.255119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.524340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.676971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.956247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.973111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.095276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.285835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.238574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.745941] LustreError: 172220:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6129.764903] LustreError: 172220:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6129.773155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.249328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.565125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.807989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.004173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.136557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.367293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.537263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.501722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.446087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.639902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.914943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.928266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.120754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.470525] LustreError: 172220:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6146.487617] LustreError: 172220:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6146.509234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.731914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.926439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.289050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.307686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.709569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.801159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.990279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.854271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.106488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.055599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.181713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.197251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.488063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.691848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.892712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.930037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.883605] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6165.888344] Lustre: Skipped 102 previous similar messages [ 6165.900848] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6165.910109] Lustre: Skipped 102 previous similar messages [ 6165.915455] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6165.922259] Lustre: Skipped 102 previous similar messages [ 6165.925838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.938536] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6165.952680] Lustre: Skipped 102 previous similar messages [ 6167.149973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.467980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.591978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.612570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.692195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.988851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.216805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.286274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.320440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.219820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.200291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.363194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.748262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.018516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.185405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.143945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.369726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.521132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.734192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.885743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.055467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.208614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.349877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.372193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.596525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.567377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.563888] LustreError: 172220:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6196.573644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.575481] LustreError: 172220:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6197.430152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.581326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.651677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.846604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.769490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.816109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.611802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.518593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.504106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.788848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.811402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.857361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.003411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.898593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.878313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.204920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.498532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.939734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.513996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.571539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.605032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.440508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.104915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.481327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.508288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.469125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.893735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.555952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.915109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.173620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.610417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.570364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.380313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.614262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.554336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.561607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.621995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.657867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.522523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.429719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.281133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.376881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.372410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.477749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.456792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.553939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.569570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.621137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.482594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.849531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.829780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.878148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.844495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.775906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.651896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.626737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.524476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.330222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.193498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.061831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.956771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.036636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.006444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.057942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.972909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.865435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.099144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.359083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.501317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.659976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.944974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.192456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.971921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.860766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.882516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.890123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.850656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.916209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.941672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.292929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.505199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.989966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.145831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.895145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.755258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.418351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.329538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.402862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.308878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.627300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.876754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.111853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.101480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.177910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.147590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.197165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.922234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.871864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.679719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.658455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.577151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.814179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.788983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.929842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.925867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.131965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.292993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.380620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.216940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.229134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.117976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.164663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.311031] LustreError: 172220:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6315.327582] LustreError: 172220:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6315.337788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.290738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.271065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.460404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6319.440291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.816641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.944180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.168588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.290529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.536759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.940084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.208986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.341535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.510802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6331.496393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.655958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.174532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.498387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.725695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.773370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.075661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.181775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.141322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.947211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6342.913820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.176394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.164892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.510386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.608921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.021642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.980749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6351.062088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.234398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.614488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6354.768937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.840123] LustreError: 172220:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6355.851621] LustreError: 172220:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6355.866326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.922087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.968781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.055587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.996455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.920912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6361.814492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6362.774539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.727684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.771967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.577320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.533754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.224223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.047807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.940347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.795337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.842126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.679603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.726682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6373.741508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.511079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.353839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.449341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6377.273951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.503962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.569326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.448034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.218104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.933544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.171420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6384.134773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6385.074692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6386.062622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6386.893205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.827966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6388.883174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6390.119405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.115397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6392.484310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.591635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6394.574700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6395.626626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6396.604200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.670041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.970039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.038907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6401.317949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.441026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6403.522789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6404.490425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.475387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6406.418929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6407.566532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6408.913221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6409.909957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.674139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.344840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6412.566960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.544601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6414.621427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.383968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.255100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.147134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6418.192531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.215292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.157218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.969768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6421.805188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6422.983301] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6422.993326] Lustre: Skipped 242 previous similar messages [ 6423.019717] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6423.026929] Lustre: Skipped 242 previous similar messages [ 6423.030061] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6423.033683] Lustre: Skipped 242 previous similar messages [ 6423.036871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6423.046859] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6423.049105] Lustre: Skipped 242 previous similar messages [ 6423.928983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.095901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.963535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6427.233154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6428.412737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6429.715329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.985735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6432.309350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6433.569186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6434.754660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.794637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6436.880621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.086161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6439.491113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6440.555595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.624826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.649753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6443.484714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6444.801641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6446.058438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6446.891428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6447.792895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6448.575437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6449.989466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6450.842189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.741661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6452.281449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6453.037395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6453.995905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6454.970558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.964396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6456.715848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6457.634120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6458.803721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6459.682942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6460.808962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6461.857460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6462.952610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6464.003400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6464.818745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6465.531850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6466.438784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6467.454493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6468.307436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6469.311713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6470.109459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6470.999981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6471.688867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6472.456127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6473.519339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6474.372070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6475.232163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6476.063497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6477.066418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6477.969765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6478.899635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6479.680473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6480.575967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6481.542515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6482.519134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6483.339953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6484.399368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6485.413145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6486.278129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6487.062853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6487.889057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6488.830527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6489.652907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6490.586838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6491.466166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6492.281462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6493.191444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6494.203531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6495.097611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6495.943400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6496.758524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6497.656988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6498.396137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6499.183903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6499.901187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6500.723866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6501.425273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6502.273604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6503.325922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6504.370777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6505.194328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6505.995738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6506.726591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6507.418487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6508.197254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6508.921858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6509.866876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6510.788213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6511.670296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6512.614048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6513.623304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6514.611260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6515.622099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6516.322528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6517.283252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6518.305906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6519.762710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6520.928635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6521.989629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6523.350315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6524.488375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6525.613293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6527.052596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6528.228746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6529.680085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6530.825459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6531.893672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6533.108836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6534.013220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6535.012695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6536.069688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6537.012816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6537.885367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6538.826562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6539.650278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6540.529879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6541.586750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6542.501662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6543.458239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6544.436990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6545.343242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6546.267709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6547.371967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6548.423087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6549.396710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6550.466429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6551.368854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6552.435887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6553.491328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6554.390422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6555.359823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6556.343577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6557.318653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6558.331098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6559.288494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6560.054467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6561.166882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6562.046439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6563.055493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6564.404138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6565.519731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6566.529192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6567.655268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6568.801386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6570.043188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6571.200889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6572.250808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6573.148946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6574.361488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6575.364603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6576.429723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6577.292969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6578.212963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6579.155284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6580.232777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6581.424941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6582.418403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6583.466325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6584.749083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6585.987374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6587.023794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6588.071323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6589.230712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6590.392064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6591.491821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6592.871127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6594.005051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6595.139491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6596.595111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6597.631233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6598.573417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6599.876185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6600.942610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6602.067690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6603.176734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6604.168758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6605.069377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6606.279698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6607.365902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6608.349232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6609.232645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6610.434403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6611.788377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6612.909735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6614.429286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6615.493202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6616.322277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6617.474322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6618.774452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6620.088638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6621.156955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6622.303574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6623.571634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6624.924645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6626.146377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6627.492558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6628.650302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6629.689404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6631.003992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6632.252715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6633.485112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6634.762189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6635.944502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6637.205043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6638.238179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6639.523473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6640.819885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6642.183726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6643.311540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6644.526024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6645.595621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6646.599835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6647.630730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6648.288849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6649.419925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6650.733662] LustreError: 172220:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6650.752348] LustreError: 172220:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6650.761902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6652.144639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6653.301944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6654.561140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6655.760145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6657.164414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6658.194847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6659.414576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6660.276918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6661.343177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6662.606185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6663.869514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6664.933843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6665.981468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6666.889264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6668.062723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6669.157196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6670.332734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6671.122757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6672.349918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6673.497737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6674.417905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6675.674749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6676.675264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6677.532744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6678.297951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6679.014902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6679.772837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6680.774658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6681.943487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6683.075658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6684.115273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6685.228268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6686.613782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6687.634976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6688.624993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6689.806055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6691.039598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6692.196098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6693.295803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6694.410191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6695.388328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6696.455784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6697.579489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6698.614428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6699.753569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6701.071604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6702.127311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6703.214719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6704.212993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6705.377495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6706.300536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6707.349254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6708.366781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6709.489945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6710.392823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6711.610459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6712.642074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6713.860410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6715.007724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6716.014690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6717.256114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6718.439651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6719.766348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6721.085218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6722.238450] LustreError: 172220:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6722.247490] LustreError: 172220:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6722.267018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6723.533764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6724.570306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6725.951435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6727.086545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6728.425593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6729.660501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6730.745524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6731.859153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6732.904682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6733.951168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6735.146966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6736.439172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6737.505679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6738.597984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6739.953062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6741.337356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6742.678508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6743.851061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6744.951737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6746.049113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6747.290668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6748.394308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6749.573425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6750.778467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6751.864573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6752.973327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6754.079374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6755.300656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6756.353664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6757.473629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6758.909781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6760.020453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6761.176606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6762.137232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6763.343063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6764.551709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6765.822458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6767.007595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6767.937192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6769.193101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6770.199736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6771.202354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6772.107269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6773.230232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6774.173893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6775.182384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6775.950927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6776.698026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6777.771617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6778.908244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6780.205555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6781.237546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6782.433313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6783.708220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6785.050557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6786.218828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6787.202271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6788.473727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6789.532984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6790.727647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6792.188794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6793.472503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6794.812351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6796.139666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6797.498325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6798.801744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6799.982182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6801.039958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6802.144424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6803.889977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6804.858606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6806.095493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6807.230749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6808.215274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6809.556782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6810.700304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6811.759391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6812.803142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6814.140217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6815.064960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6816.246363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6817.475809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6818.712992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6819.817927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6820.804646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6821.778116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6822.802413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6823.701834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6824.776601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6826.100914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6827.061769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6827.841838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6828.549304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6829.206549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6829.960338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6831.015168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6831.832327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6833.131132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6834.240312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6835.410498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6836.276879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6837.011478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6837.694694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6838.480789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6839.133362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6840.255342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6841.330778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6842.213561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6843.189584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6843.982655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6844.925507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6845.756234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6846.565838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6847.553579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6848.587759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6849.689976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6850.824972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6851.967936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6853.160364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6854.260114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6855.474273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6856.770497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6857.834712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6859.052843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6860.238103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6861.298314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6862.165243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6863.121803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6864.108587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6864.974209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6866.043359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6867.238876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6868.216964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6869.397258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6870.371189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6871.504941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6872.621838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6873.819562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6875.066267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6876.007878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6877.133325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6878.191531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6879.147160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6880.231634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6881.334567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6882.327761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6883.418435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6884.541837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6885.866319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6886.797874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6888.036425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6889.023745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6890.213513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6891.469069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6892.438063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6893.540804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6894.350111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6895.565899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6896.400377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6897.443567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6898.496621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6899.485247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6900.579490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6901.676487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6902.422474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6903.385855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6904.539546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6905.698329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6906.905540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6907.843222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6908.546154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6909.308062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6909.921471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6910.713543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6911.750518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6913.048456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6914.375230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6915.731888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6917.170824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6918.755481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6920.367775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6922.063275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6923.793569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6925.474067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6926.955530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6929.180162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6931.055888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6932.831757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6934.357220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6935.908717] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6935.918343] Lustre: Skipped 475 previous similar messages [ 6935.948703] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6935.959896] Lustre: Skipped 475 previous similar messages [ 6935.966862] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6935.970783] Lustre: Skipped 475 previous similar messages [ 6935.977766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6936.000597] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6936.021171] Lustre: Skipped 475 previous similar messages [ 6937.872131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6939.062427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6940.755928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6942.356708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6944.220655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6945.593271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6946.617807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6948.058551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6949.355253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6950.466405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6951.668963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6952.814798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6953.867466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6955.083355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6956.116283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6957.350349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6958.632082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6960.297213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6961.597824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6962.920708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6964.061110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6965.061388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6966.253684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6967.612630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6968.762141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6969.895725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6971.184739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6972.175753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6973.437429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6974.594169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6975.991836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6977.200297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6978.417178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6979.518352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6980.712470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6981.790851] LustreError: 172220:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6981.807461] LustreError: 172220:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6981.834489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6983.053501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6984.270221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6985.498226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6986.480189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6987.710147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6988.915126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6990.223987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6991.505903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6992.510203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6993.818399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6995.067668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6996.441238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6997.554422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6998.654323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6999.694317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7000.708140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7002.195845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7003.505594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7005.018691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7006.344886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7007.399882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7008.638275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7009.732327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7010.825034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7012.052489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7013.412049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7014.521826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7015.548063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7016.835442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7018.135362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7019.207695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7492.949435] LustreError: 174026:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 7492.963599] LustreError: 172220:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 8167.398536] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8167.410708] Lustre: Skipped 67 previous similar messages [ 8167.415430] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8171.555756] Lustre: server umount lustre-OST0000 complete [ 8177.633225] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8177.641044] Lustre: Skipped 1 previous similar message [ 8182.753369] LustreError: 212773:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8182.767967] LustreError: 212773:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8182.872710] Lustre: server umount lustre-MDT0000 complete [ 8186.941428] LustreError: 172182:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787556334 with bad export cookie 12496555490177490434 [ 8186.951815] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8186.970957] LustreError: 172182:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8187.290357] Lustre: server umount lustre-MDT0001 complete [ 8199.250107] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing unload_modules_local [ 8202.689339] Key type lgssc unregistered [ 8202.974229] LNet: 227535:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8202.977091] LNetError: 227535:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8202.995304] LNet: Removed LNI 192.168.201.115@tcp [ 8203.808672] Key type .llcrypt unregistered [ 8203.811585] Key type ._llcrypt unregistered [ 8230.577766] Key type ._llcrypt registered [ 8230.585101] Key type .llcrypt registered [ 8230.692229] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 8232.289183] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8232.328034] alg: No test for adler32 (adler32-zlib) [ 8233.611820] Lustre: Lustre: Build Version: 2.17.57_83_g6b3cfd8 [ 8233.989567] LNet: Added LNI 192.168.201.115@tcp [8/256/0/180] [ 8235.767265] Key type lgssc registered [ 8237.134800] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8246.044970] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8246.059508] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8247.609604] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8253.555326] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 8258.020715] LustreError: 228952:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8258.044295] LustreError: 228952:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8263.145905] LustreError: 228951:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8263.921537] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8264.383811] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8269.899747] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 8278.839027] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8284.952476] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8292.072214] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8292.372826] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8297.454725] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 8299.526280] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 8307.685512] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8313.314349] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 8315.447846] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8315.457243] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 8315.464292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8315.471327] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 8315.472890] LustreError: 228971:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 8317.861427] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8317.871132] Lustre: Skipped 1 previous similar message [ 8317.871505] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8320.421861] Lustre: lustre-OST0000: Not available for connect from 192.168.201.15@tcp (stopping) [ 8320.424827] Lustre: Skipped 1 previous similar message [ 8323.045912] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8323.050830] Lustre: Skipped 1 previous similar message [ 8323.832408] Lustre: server umount lustre-OST0000 complete [ 8335.575388] LustreError: 228971:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 8360.134422] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8360.692741] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8369.275653] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 8373.996638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8374.018046] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8374.025275] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 8378.249286] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8379.969668] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 8386.636945] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8e83445b6000.ost_server_uuid 50 [ 8388.488640] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8e83445b6000.ost_server_uuid in IDLE state after 0 sec [ 8393.699735] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8393.708966] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8393.723544] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8394.731421] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8398.276503] Lustre: server umount lustre-OST0000 complete [ 8402.913344] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8402.918986] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8402.933878] Lustre: Skipped 1 previous similar message [ 8402.947462] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8402.957938] Lustre: Skipped 2 previous similar messages [ 8408.943307] Lustre: server umount lustre-MDT0000 complete [ 8410.081136] LustreError: 228952:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8410.122669] LustreError: 228952:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8414.186459] LustreError: 231262:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8414.213449] LustreError: 231262:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8414.353602] LustreError: 228933:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787556562 with bad export cookie 16572073366130732355 [ 8414.366453] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8414.370537] LustreError: 228933:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8414.758979] Lustre: server umount lustre-MDT0001 complete [ 8428.471849] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing unload_modules_local [ 8432.513888] Key type lgssc unregistered [ 8432.858710] LNet: 234166:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8432.866850] LNetError: 234166:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8432.882787] LNet: Removed LNI 192.168.201.115@tcp [ 8433.858333] Key type .llcrypt unregistered [ 8433.860683] Key type ._llcrypt unregistered [ 8474.110260] Key type ._llcrypt registered [ 8474.115564] Key type .llcrypt registered [ 8474.255622] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 8475.396126] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8475.571858] alg: No test for adler32 (adler32-zlib) [ 8476.759830] Lustre: Lustre: Build Version: 2.17.57_83_g6b3cfd8 [ 8477.031900] LNet: Added LNI 192.168.201.115@tcp [8/256/0/180] [ 8478.703188] Key type lgssc registered [ 8480.128572] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8490.193666] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8490.213335] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8491.766981] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8496.162440] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 8501.224457] LustreError: 235900:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8501.235624] LustreError: 235900:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8505.789051] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8506.206554] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8511.201466] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 8514.743157] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 8521.444235] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8521.788049] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8528.351203] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 8536.192228] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 8537.765244] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8537.948520] Lustre: lustre-OST0001: new disk, initializing [ 8537.953818] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8537.959100] Lustre: 238208:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8538.061882] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8543.880785] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8543.897883] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8543.986962] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8544.458596] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 8556.494819] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8564.130235] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 8580.081300] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8580.112468] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8580.121375] Lustre: Skipped 3 previous similar messages [ 8580.137568] Lustre: Skipped 3 previous similar messages [ 8584.569288] Lustre: server umount lustre-MDT0000 complete [ 8585.186434] LustreError: 235900:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8585.205257] LustreError: 235900:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8589.011688] LustreError: 236635:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787556736 with bad export cookie 3309138727379344152 [ 8589.020275] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8589.410438] Lustre: server umount lustre-MDT0001 complete [ 8603.894570] Lustre: server umount lustre-OST0000 complete [ 8606.508604] Lustre: 235163:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787556738/real 1787556738] req@ffff98ee653bca80 x1874388974659456/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787556754 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8606.530165] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8608.445715] Lustre: server umount lustre-OST0001 complete [ 8631.363856] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 8644.230619] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8644.648245] LustreError: 241317:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8644.717871] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8649.988633] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 8650.217447] LustreError: 241318:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8654.310843] LustreError: 241317:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8659.217910] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8659.425358] Lustre: mylustre-MDT0001: Not available for connect from 0@lo (not set up) [ 8659.706158] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8663.868720] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 8666.988913] Lustre: 242460:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8674.683373] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8683.781807] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 8685.545402] LustreError: 242814:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8690.737137] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49760) [ 8694.222346] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8694.587417] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8694.597081] Lustre: Skipped 1 previous similar message [ 8702.142176] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 8710.766182] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8714.791107] Lustre: 244333:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8730.607795] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8730.646636] Lustre: Skipped 1 previous similar message [ 8730.669615] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8730.683448] Lustre: Skipped 3 previous similar messages [ 8735.726893] LustreError: 241318:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8735.741593] LustreError: 241318:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 8735.811332] Lustre: server umount mylustre-MDT0000 complete [ 8741.351609] LustreError: 243643:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787556889 with bad export cookie 3309138727379346952 [ 8741.353953] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8741.383382] LustreError: 243643:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 8742.258511] Lustre: server umount mylustre-MDT0001 complete [ 8761.141894] Lustre: server umount mylustre-OST0000 complete [ 8761.825060] Lustre: 235162:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787556893/real 1787556893] req@ffff98ee4975a680 x1874388974721920/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787556909 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8761.863973] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8761.877264] Lustre: Skipped 2 previous similar messages [ 8766.432727] Lustre: 235164:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787556898/real 1787556898] req@ffff98ee643a4000 x1874388974722432/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787556914 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8766.510500] Lustre: 235164:0:(client.c:2503:ptlrpc_expire_one_request()) Skipped 1 previous similar message [ 8767.433113] Lustre: server umount mylustre-OST0001 complete [ 8794.676081] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 8806.942028] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8807.660337] LustreError: 246642:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8807.693327] LustreError: 246642:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 8807.794962] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8812.870115] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 8822.575706] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8828.803358] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 8832.088259] Lustre: 247787:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8840.426812] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8848.104186] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 8852.232135] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49792) [ 8857.583115] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8865.033575] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 8873.396605] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8877.021436] Lustre: 249660:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8887.851617] Lustre: server umount tfs-MDT0000 complete [ 8888.822842] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8888.834994] LustreError: 248733:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8888.838270] Lustre: Skipped 2 previous similar messages [ 8888.876026] LustreError: 248733:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [ 8892.234580] LustreError: 249663:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787557040 with bad export cookie 3309138727379348625 [ 8892.241614] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8892.248228] LustreError: 249663:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8892.584484] Lustre: server umount tfs-MDT0001 complete [ 8907.608268] Lustre: server umount tfs-OST0000 complete [ 8909.472471] Lustre: 235161:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787557041/real 1787557041] req@ffff98ee4975bb80 x1874388974775040/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787557057 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8909.514610] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8909.533152] Lustre: Skipped 1 previous similar message [ 8911.865181] Lustre: server umount tfs-OST0001 complete [ 8941.917562] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 8955.741940] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8956.198908] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8956.202849] Lustre: Skipped 3 previous similar messages [ 8960.219972] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 8970.821672] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8977.687963] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 8981.614372] Lustre: 253053:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8990.014480] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8998.811934] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 9001.686895] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49824) [ 9009.172851] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9017.432389] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 9026.119381] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9030.160203] Lustre: 254923:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 9037.070524] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 03:39:43 (1787557183) [ 9055.714062] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9055.725130] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9055.748891] Lustre: Skipped 3 previous similar messages [ 9055.763251] Lustre: Skipped 3 previous similar messages [ 9060.853017] LustreError: 251907:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9060.885157] LustreError: 251907:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 10 previous similar messages [ 9061.192688] Lustre: server umount lustre-MDT0000 complete [ 9065.474389] LustreError: 251887:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787557213 with bad export cookie 3309138727379350067 [ 9065.494783] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9065.960628] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9072.337409] Lustre: server umount lustre-MDT0001 complete [ 9082.994290] Lustre: server umount lustre-OST0000 complete [ 9093.888583] Lustre: server umount lustre-OST0001 complete [ 9102.495404] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_hostid [ 9113.051308] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 9156.552050] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9156.765404] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9156.794332] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9156.908686] Lustre: lustre-MDT0000: new disk, initializing [ 9157.000834] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 9157.009194] Lustre: Skipped 3 previous similar messages [ 9157.035018] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9162.508408] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 9174.739546] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9174.957468] Lustre: 258113:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9175.043146] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9175.048083] Lustre: Skipped 1 previous similar message [ 9175.175266] Lustre: lustre-MDT0001: new disk, initializing [ 9175.377905] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9175.394988] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9180.664979] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 9191.933437] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9192.326935] Lustre: lustre-OST0000: new disk, initializing [ 9192.336427] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9192.346790] Lustre: 259696:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9194.142087] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9194.162443] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9194.328083] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9199.454525] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 9210.907518] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9211.087186] Lustre: lustre-OST0001: new disk, initializing [ 9211.090503] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 9211.095769] Lustre: 260568:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9212.444398] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 9212.486500] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 9212.703705] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 9218.685395] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 9235.466519] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 03:43:01 (1787557381) [ 9255.777833] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 03:43:22 (1787557402) [ 9271.268607] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9271.271649] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9271.272408] LustreError: Skipped 1 previous similar message [ 9271.286338] Lustre: Skipped 2 previous similar messages [ 9271.290630] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9271.297885] Lustre: Skipped 4 previous similar messages [ 9277.576170] Lustre: server umount lustre-OST0000 complete [ 9283.045092] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9288.935057] Lustre: server umount lustre-MDT0000 complete [ 9293.980571] LustreError: 260567:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787557441 with bad export cookie 3309138727379351397 [ 9293.991487] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9293.994894] LustreError: 260567:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 9294.834987] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9294.841238] Lustre: Skipped 6 previous similar messages [ 9300.635920] Lustre: server umount lustre-MDT0001 complete [ 9316.259560] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing unload_modules_local [ 9392.513116] Lustre: server umount lustre-OST0001 complete [ 9400.997617] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_hostid [ 9420.401306] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 9461.419904] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9461.781316] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9461.828758] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9461.926750] Lustre: lustre-MDT0000: new disk, initializing [ 9462.066384] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9462.075108] Lustre: Skipped 3 previous similar messages [ 9462.098259] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9467.923490] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 9479.720919] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9479.879394] Lustre: 265431:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9479.951968] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9479.955711] Lustre: Skipped 1 previous similar message [ 9480.030723] Lustre: lustre-MDT0001: new disk, initializing [ 9480.207622] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9480.222606] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9485.506675] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 9496.374995] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9503.259244] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9511.302582] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9511.676567] Lustre: lustre-OST0000: new disk, initializing [ 9511.684719] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9511.692234] Lustre: 267398:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9513.458678] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9513.475791] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9513.582399] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9518.627804] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 9529.835834] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9544.172167] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9544.184868] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9544.205051] Lustre: Skipped 5 previous similar messages [ 9544.222679] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9544.241582] Lustre: Skipped 2 previous similar messages [ 9545.874925] Lustre: server umount lustre-OST0000 complete [ 9551.841831] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9551.845199] LustreError: Skipped 1 previous similar message [ 9558.116755] Lustre: server umount lustre-MDT0000 complete [ 9559.522214] LustreError: 265437:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9559.543795] LustreError: 265437:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 15 previous similar messages [ 9563.139382] LustreError: 265421:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787557710 with bad export cookie 3309138727379352867 [ 9563.152671] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9563.157306] LustreError: 265421:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9563.816800] Lustre: server umount lustre-MDT0001 complete [ 9577.793740] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing unload_modules_local [ 9582.020967] Key type lgssc unregistered [ 9582.331740] LNet: 269163:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9582.345471] LNetError: 269163:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9582.387312] LNet: Removed LNI 192.168.201.115@tcp [ 9583.789647] Key type .llcrypt unregistered [ 9583.791710] Key type ._llcrypt unregistered [ 9619.067154] Key type ._llcrypt registered [ 9619.069801] Key type .llcrypt registered [ 9619.333496] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 9621.446287] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9621.491868] alg: No test for adler32 (adler32-zlib) [ 9622.645512] Lustre: Lustre: Build Version: 2.17.57_83_g6b3cfd8 [ 9622.910464] LNet: Added LNI 192.168.201.115@tcp [8/256/0/180] [ 9624.616929] Key type lgssc registered [ 9626.113263] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9637.134903] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9637.184679] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9638.975324] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9644.760733] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 9649.121620] LustreError: 270631:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9649.154920] LustreError: 270631:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 9654.251889] LustreError: 270632:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9655.556183] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9656.193480] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9662.622429] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 9671.810190] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9678.876691] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9687.729547] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9688.152835] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9696.860874] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 9705.855593] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9707.314367] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 9708.687279] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 9711.072873] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9711.086349] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9711.102213] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9714.146399] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9714.167688] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9714.169788] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9716.016601] Lustre: server umount lustre-OST0000 complete [ 9722.849903] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9722.860444] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9722.884923] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9722.889119] Lustre: Skipped 1 previous similar message [ 9727.266162] Lustre: server umount lustre-MDT0000 complete [ 9729.509437] LustreError: 270632:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9729.545318] LustreError: 270632:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9732.020646] LustreError: 270613:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787557879 with bad export cookie 16330349327571596308 [ 9732.032929] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9732.037452] LustreError: 270613:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9732.851303] Lustre: server umount lustre-MDT0001 complete [ 9746.647441] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing unload_modules_local [ 9751.029681] Key type lgssc unregistered [ 9751.600795] LNet: 274219:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9751.622412] LNetError: 274219:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9751.645794] LNet: Removed LNI 192.168.201.115@tcp [ 9752.908357] Key type .llcrypt unregistered [ 9752.913837] Key type ._llcrypt unregistered [ 9773.077630] Key type ._llcrypt registered [ 9773.080404] Key type .llcrypt registered [ 9773.264093] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 9774.789283] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9774.844260] alg: No test for adler32 (adler32-zlib) [ 9776.441485] Lustre: Lustre: Build Version: 2.17.57_83_g6b3cfd8 [ 9776.815292] LNet: Added LNI 192.168.201.115@tcp [8/256/0/180] [ 9778.551721] Key type lgssc registered [ 9780.109282] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9791.713377] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9791.747746] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9807.520482] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 9812.639622] LustreError: 275413:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.115@tcp: failed processing log, type 4: rc = -110 [ 9839.327217] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9846.462824] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [ 9853.952860] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9856.475893] Lustre: server umount lustre-OST0000 complete [ 9878.339503] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing unload_modules_local [ 9886.464985] Key type lgssc unregistered [ 9886.785170] LNet: 277048:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9886.805240] LNetError: 277048:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9886.826241] LNet: Removed LNI 192.168.201.115@tcp [ 9887.714468] Key type .llcrypt unregistered [ 9887.722042] Key type ._llcrypt unregistered [ 9941.435140] Key type ._llcrypt registered [ 9941.437645] Key type .llcrypt registered [ 9941.554659] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [ 9942.723043] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9942.820878] alg: No test for adler32 (adler32-zlib) [ 9944.418882] Lustre: Lustre: Build Version: 2.17.57_83_g6b3cfd8 [ 9944.755573] LNet: Added LNI 192.168.201.115@tcp [8/256/0/180] [ 9946.439217] Key type lgssc registered [ 9947.903244] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9965.834975] loop0: detected capacity change from 0 to 204800000 [ 9966.266633] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9966.289302] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9966.931924] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9968.323253] loop0: detected capacity change from 0 to 204800000 [ 9968.607276] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9968.614981] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9969.082838] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9970.728471] loop0: detected capacity change from 0 to 204800000 [ 9971.106957] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9971.134703] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9971.344563] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9973.042318] loop0: detected capacity change from 0 to 204800000 [ 9973.348591] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9973.370833] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9973.671319] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9975.581793] loop0: detected capacity change from 0 to 204800000 [ 9975.637075] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9981.359693] loop0: detected capacity change from 0 to 204800000 [ 9981.456222] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9985.829280] loop0: detected capacity change from 0 to 204800000 [ 9985.869572] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9990.411657] loop0: detected capacity change from 0 to 204800000 [ 9990.449318] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9996.030528] loop0: detected capacity change from 0 to 204800000 [ 9996.180718] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9996.241291] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9996.321869] Lustre: 280469:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9996.327898] Lustre: 280469:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 9996.332484] Lustre: 280469:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 9996.336590] Lustre: 280469:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 9996.340408] Lustre: 280469:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9996.344646] Lustre: 280469:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 9996.348695] Lustre: 280469:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9996.352757] Lustre: 280469:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9996.359113] Lustre: 280469:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 9996.366475] Lustre: 280469:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 9996.386520] Lustre: 280469:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 9996.397596] Lustre: 280469:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 9996.404790] Lustre: 280469:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 9996.411797] Lustre: 280469:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [10004.117111] Lustre: server umount MGS complete [10005.473318] loop0: detected capacity change from 0 to 204800000 [10005.518555] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10006.143633] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10006.267801] Lustre: lustre-MDT0000: Aborting client recovery [10007.613060] loop1: detected capacity change from 0 to 204800000 [10007.838234] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10007.962595] Lustre: 280903:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [10007.980123] Lustre: 280903:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [10008.024723] Lustre: 280903:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [10008.040739] Lustre: 280903:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [10008.073762] Lustre: 280903:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [10008.096265] Lustre: 280903:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [10008.114611] Lustre: 280903:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [10008.163639] Lustre: Found index 1 for lustre-MDT0001, updating log [10008.581242] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10008.664338] Lustre: lustre-MDT0001: Aborting client recovery [10010.478580] loop2: detected capacity change from 0 to 204800000 [10010.749485] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10010.876366] Lustre: 280982:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [10010.897088] Lustre: 280982:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [10010.922565] Lustre: 280982:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [10010.940170] Lustre: 280982:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [10010.965055] Lustre: 280982:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [10010.998472] Lustre: 280982:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [10011.011592] Lustre: 280982:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [10011.239619] Lustre: Found index 0 for lustre-OST0000, updating log [10011.319727] Lustre: 281002:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10011.455314] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10011.475469] Lustre: lustre-OST0000: Aborting recovery [10012.956480] loop3: detected capacity change from 0 to 204800000 [10013.164600] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10013.379537] Lustre: 281071:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [10013.402076] Lustre: 281071:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [10013.413546] Lustre: 281071:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [10013.421350] Lustre: 281071:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [10013.436604] Lustre: 281071:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [10013.444641] Lustre: 281071:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [10013.451994] Lustre: 281071:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [10013.492637] Lustre: Found index 1 for lustre-OST0001, updating log [10013.538668] Lustre: 281076:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10013.648095] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10013.684417] Lustre: lustre-OST0001: Aborting recovery [10013.685790] LustreError: 280847:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -108 [10015.660183] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [10015.670960] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [10015.721629] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [10015.753056] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [10018.730375] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [10018.735843] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [10022.293476] LustreError: 280814:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 3552us: rc = -2 [10022.906369] LustreError: 280814:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 591us: rc = -2 [10022.920239] LustreError: 280814:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 22 previous similar messages [10039.265066] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10039.267055] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10039.293784] Lustre: Skipped 2 previous similar messages [10039.308718] Lustre: Skipped 1 previous similar message [10042.103909] Lustre: server umount lustre-MDT0000 complete [10043.324582] LustreError: 280798:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787558191 with bad export cookie 9281470186008220403 [10043.355391] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10044.385041] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10044.388797] LustreError: 281339:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10044.405952] Lustre: Skipped 1 previous similar message [10044.410107] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10044.410119] Lustre: Skipped 2 previous similar messages [10044.449242] LustreError: 281339:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10048.417769] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10048.417856] LustreError: 280811:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10048.425110] Lustre: Skipped 2 previous similar messages [10048.452209] LustreError: 280811:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10050.029376] Lustre: server umount lustre-MDT0001 complete [10065.375239] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10065.501063] Lustre: server umount lustre-OST0000 complete [10072.952867] Lustre: server umount lustre-OST0001 complete [10081.970248] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 03:57:08 (1787558228) [10115.640720] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_hostid [10126.426932] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [10170.761414] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10171.056184] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10171.099862] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10171.225566] Lustre: lustre-MDT0000: new disk, initializing [10171.318707] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10171.348390] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10176.998696] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [10188.079735] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10188.251685] Lustre: 285147:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10188.307806] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10188.310944] Lustre: Skipped 1 previous similar message [10188.415842] Lustre: lustre-MDT0001: new disk, initializing [10188.573268] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10188.612861] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10188.633071] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10193.997707] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [10204.356784] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10211.466091] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10220.694697] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10221.010674] Lustre: lustre-OST0000: new disk, initializing [10221.015972] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10221.031903] Lustre: 287118:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10221.177076] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10228.885411] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [10229.387705] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10229.404701] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10229.524126] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10240.677295] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10247.013766] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10260.126165] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10268.278414] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10275.328698] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [10321.898709] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10321.915047] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10321.933163] Lustre: Skipped 1 previous similar message [10321.939867] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10321.944600] Lustre: Skipped 1 previous similar message [10324.582854] Lustre: server umount lustre-OST0000 complete [10332.134605] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10332.137096] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10332.154635] Lustre: Skipped 1 previous similar message [10332.156138] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10332.164968] LustreError: Skipped 1 previous similar message [10332.192791] Lustre: Skipped 1 previous similar message [10341.348040] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10341.354636] Lustre: Skipped 3 previous similar messages [10342.367534] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10342.695285] Lustre: server umount lustre-MDT0000 complete [10346.241590] LustreError: 285141:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787558494 with bad export cookie 9281470186008223224 [10346.251688] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10346.254698] LustreError: 285141:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [10346.621944] Lustre: server umount lustre-MDT0001 complete [10356.713857] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10362.005908] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [10367.250490] Lustre: server umount MGS complete [10381.417216] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10382.051917] LustreError: 291006:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10382.089468] LustreError: 291019:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10382.196042] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10386.540702] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [10387.428494] LustreError: 291007:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10391.524850] LustreError: 291006:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10396.229195] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10401.291796] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [10410.846886] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10419.234167] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10427.766210] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10436.339611] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [10445.603584] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10487.788896] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10487.794214] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10487.809380] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10487.814079] Lustre: Skipped 2 previous similar messages [10487.829051] Lustre: Skipped 1 previous similar message [10492.807101] Lustre: server umount lustre-OST0000 complete [10498.026843] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10503.040558] Lustre: server umount lustre-MDT0000 complete [10503.138456] LustreError: 291736:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10503.159140] LustreError: 291736:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10507.516436] LustreError: 293867:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787558655 with bad export cookie 9281470186008225177 [10507.526636] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10507.529520] LustreError: 293867:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10507.910308] Lustre: server umount lustre-MDT0001 complete [10519.497771] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing unload_modules_local [10523.270381] Key type lgssc unregistered [10523.610800] LNet: 294978:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10523.618420] LNetError: 294978:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10524.651028] LNet: Removed LNI 192.168.201.115@tcp [10525.841856] Key type .llcrypt unregistered [10525.844723] Key type ._llcrypt unregistered [10572.801215] Key type ._llcrypt registered [10572.802877] Key type .llcrypt registered [10572.878832] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_hostid [10591.630541] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [10592.533288] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10592.568708] alg: No test for adler32 (adler32-zlib) [10593.767857] Lustre: Lustre: Build Version: 2.17.57_83_g6b3cfd8 [10594.031575] LNet: Added LNI 192.168.201.115@tcp [8/256/0/180] [10595.815293] Key type lgssc registered [10597.353335] Lustre: Echo OBD driver; http://www.lustre.org/ [10637.630684] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10637.672517] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10638.960268] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10638.991541] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10639.073201] Lustre: lustre-MDT0000: new disk, initializing [10639.151756] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10639.166978] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10642.866884] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [10655.220959] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10655.427431] Lustre: 298937:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10655.566240] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10655.573220] Lustre: Skipped 1 previous similar message [10655.722325] Lustre: lustre-MDT0001: new disk, initializing [10655.882972] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10655.949388] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10655.967976] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10661.134851] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [10671.648138] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10679.248298] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10689.052196] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10689.522441] Lustre: lustre-OST0000: new disk, initializing [10689.526916] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10689.533697] Lustre: 300905:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10689.801935] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10697.294143] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10697.310253] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10697.427235] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10698.440278] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [10708.920328] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10716.652136] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10727.151500] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10733.397080] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10737.853601] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [10784.748914] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10784.749208] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10784.764347] LustreError: Skipped 1 previous similar message [10784.765558] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10784.805755] Lustre: Skipped 1 previous similar message [10787.133774] Lustre: server umount lustre-OST0000 complete [10793.953438] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10793.958605] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10793.967644] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10793.970970] Lustre: Skipped 1 previous similar message [10794.978071] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10794.979769] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10795.020304] Lustre: Skipped 1 previous similar message [10797.589631] Lustre: server umount lustre-MDT0000 complete [10801.157995] LustreError: 298929:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787558949 with bad export cookie 1953409374393264840 [10801.162215] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10801.166592] LustreError: 298929:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10801.488811] Lustre: server umount lustre-MDT0001 complete [10810.172775] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10815.228638] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [10820.336900] Lustre: server umount MGS complete [10832.543896] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10833.104428] LustreError: 304729:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10833.122166] LustreError: 304729:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10833.132843] LustreError: 304742:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10833.212359] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10838.369849] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [10838.501764] LustreError: 304730:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10842.598325] LustreError: 304729:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10847.277560] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10847.675623] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10853.525376] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [10863.165832] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10870.680632] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10878.498747] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10878.942904] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10885.313655] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [10894.467387] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10938.343203] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10938.344328] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10938.354377] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10938.365971] LustreError: Skipped 1 previous similar message [10938.383308] Lustre: Skipped 1 previous similar message [10943.462951] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10943.468823] Lustre: Skipped 1 previous similar message [10951.135180] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10951.402552] Lustre: server umount lustre-OST0000 complete [10955.233800] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10955.248777] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10955.268693] Lustre: Skipped 1 previous similar message [10955.278295] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10955.293789] Lustre: Skipped 2 previous similar messages [10961.555342] Lustre: server umount lustre-MDT0000 complete [10963.941730] LustreError: 305671:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10963.960054] LustreError: 305671:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10965.942195] LustreError: 304710:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787559113 with bad export cookie 1953409374393266779 [10965.954642] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10966.385520] Lustre: server umount lustre-MDT0001 complete [10977.774650] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing unload_modules_local [10980.831202] Key type lgssc unregistered [10981.106616] LNet: 308705:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10981.111904] LNetError: 308705:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10981.128836] LNet: Removed LNI 192.168.201.115@tcp [10982.049051] Key type .llcrypt unregistered [10982.051094] Key type ._llcrypt unregistered [11007.003191] Key type ._llcrypt registered [11007.008977] Key type .llcrypt registered [11007.195581] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_hostid [11022.335407] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [11023.799064] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11023.858075] alg: No test for adler32 (adler32-zlib) [11024.986871] Lustre: Lustre: Build Version: 2.17.57_83_g6b3cfd8 [11025.444413] LNet: Added LNI 192.168.201.115@tcp [8/256/0/180] [11027.303328] Key type lgssc registered [11029.309625] Lustre: Echo OBD driver; http://www.lustre.org/ [11061.111063] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [11079.357522] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11079.382435] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11080.699368] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11080.719817] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11080.783401] Lustre: lustre-MDT0000: new disk, initializing [11080.867618] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11080.880204] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11086.365048] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [11097.748860] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11098.170595] Lustre: lustre-OST0000: new disk, initializing [11098.173144] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11098.175154] Lustre: Skipped 1 previous similar message [11098.179567] Lustre: 312760:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11098.255487] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11104.323032] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [11104.333769] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [11104.380370] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [11105.184345] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [12145.450329] Lustre: server umount lustre-MDT0000 complete [12151.499376] LustreError: 311947:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787560299 with bad export cookie 6068550439915292592 [12151.528762] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12162.169604] Lustre: server umount lustre-OST0000 complete [12189.460444] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_hostid [12202.296683] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [12250.672463] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12251.041471] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12251.107604] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12251.258497] Lustre: lustre-MDT0000: new disk, initializing [12251.327871] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12251.349026] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12256.626575] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [12271.133585] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12271.418571] Lustre: 317128:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12271.610983] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12271.620095] Lustre: Skipped 1 previous similar message [12271.761148] Lustre: lustre-MDT0001: new disk, initializing [12271.928928] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12271.970673] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12271.985826] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12278.052815] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [12290.351849] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12296.720988] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12304.036086] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12304.300108] Lustre: lustre-OST0000: new disk, initializing [12304.307803] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12304.316329] Lustre: 319099:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12304.409619] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12310.588388] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12310.598042] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12310.756947] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12310.824290] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [12320.847599] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12327.839627] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12328.110760] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12334.792369] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [12335.084678] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12341.732761] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12341.753066] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12341.758913] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12341.782669] Lustre: Skipped 1 previous similar message [12344.140877] Lustre: server umount lustre-OST0000 complete [12349.408980] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12349.422696] LustreError: Skipped 1 previous similar message [12349.424531] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12349.443834] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12349.453545] Lustre: Skipped 1 previous similar message [12351.968252] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12351.972609] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12354.900635] Lustre: server umount lustre-MDT0000 complete [12358.830601] LustreError: 317119:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787560506 with bad export cookie 6068550439918233411 [12358.866718] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12358.868696] LustreError: 317119:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12359.264830] Lustre: server umount lustre-MDT0001 complete [12363.038981] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [12364.974315] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 04:35:11 (1787560511) [12372.768085] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12373.227665] LustreError: 321412:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12373.244787] LustreError: 321412:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12373.294249] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12377.840165] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [12378.595236] LustreError: 321413:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12383.718244] LustreError: 321412:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12387.329876] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12387.679636] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12393.401902] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [12402.420299] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12409.279620] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12416.944217] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12417.459608] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12424.978925] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [12433.772652] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12441.982032] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12442.166564] Lustre: lustre-OST0001: new disk, initializing [12442.178227] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [12442.188601] Lustre: 324249:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12442.335699] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12448.529915] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [12451.527156] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [12451.542225] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [12451.655764] LustreError: 324711:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12451.663944] LustreError: 324711:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [12452.720456] LustreError: 324711:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12452.741366] LustreError: 324711:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [12453.742458] LustreError: 324716:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [12455.848559] LustreError: 324716:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [12455.861609] LustreError: 324716:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 4 previous similar messages [12459.954965] LustreError: 324711:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12459.972225] LustreError: 324711:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 6 previous similar messages [12461.855098] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12462.241577] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12467.889115] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8e8348ebc000.ost_server_uuid 50 [12468.196105] LustreError: 324711:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12468.209502] LustreError: 324711:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [12469.619911] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8e8348ebc000.ost_server_uuid in FULL state after 0 sec [12476.414856] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8e8348ebc000.ost_server_uuid 50 [12478.440514] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8e8348ebc000.ost_server_uuid in IDLE state after 0 sec [12480.636924] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [12496.324619] Lustre: lustre-OST0001: Not available for connect from 192.168.201.15@tcp (stopping) [12496.332957] Lustre: Skipped 1 previous similar message [12496.354475] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12496.357306] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12496.368851] LustreError: Skipped 1 previous similar message [12496.418062] Lustre: Skipped 1 previous similar message [12499.810065] Lustre: server umount lustre-OST0001 complete [12501.371750] LustreError: 323234:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.201.15@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12508.143269] LustreError: 324712:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12508.161345] LustreError: 324712:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12509.672466] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12509.689035] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12509.711475] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12509.723341] Lustre: Skipped 4 previous similar messages [12514.564505] Lustre: server umount lustre-OST0000 complete [12520.932697] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12520.952325] LustreError: Skipped 1 previous similar message [12520.956315] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12520.973159] Lustre: Skipped 1 previous similar message [12520.989070] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12521.000294] Lustre: Skipped 3 previous similar messages [12525.236420] Lustre: server umount lustre-MDT0000 complete [12528.617302] LustreError: 321413:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12528.636648] LustreError: 321413:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12528.973873] LustreError: 321393:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787560676 with bad export cookie 6068550439918234293 [12528.982683] LustreError: 321393:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12528.990394] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12529.409295] Lustre: server umount lustre-MDT0001 complete [12541.937350] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing unload_modules_local [12545.135407] Key type lgssc unregistered [12545.556989] LNet: 326943:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12545.562397] LNetError: 326943:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12545.575904] LNet: Removed LNI 192.168.201.115@tcp [12546.419231] Key type .llcrypt unregistered [12546.423599] Key type ._llcrypt unregistered [12571.753317] Key type ._llcrypt registered [12571.755637] Key type .llcrypt registered [12571.876947] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [12573.353707] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12573.408876] alg: No test for adler32 (adler32-zlib) [12574.533796] Lustre: Lustre: Build Version: 2.17.57_83_g6b3cfd8 [12574.866213] LNet: Added LNI 192.168.201.115@tcp [8/256/0/180] [12576.671220] Key type lgssc registered [12578.020656] Lustre: Echo OBD driver; http://www.lustre.org/ [12586.102222] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12586.118530] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12587.768185] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12592.926483] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [12597.219310] LustreError: 328369:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12597.232913] LustreError: 328369:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12601.666459] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12602.355490] LustreError: 328370:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12607.467521] LustreError: 328369:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12609.417047] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12609.902933] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12615.443653] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [12626.700244] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12626.976258] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12633.130707] LustreError: 330052:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12633.147049] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [12633.928136] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [12640.243298] LustreError: 330354:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12640.258671] LustreError: 330354:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12642.793148] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12650.404122] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12650.467948] LustreError: 330050:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12650.492643] LustreError: 330050:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12650.613253] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12656.145904] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [12658.222622] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [12667.394677] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12675.562768] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [12675.798851] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12778.996537] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12779.006791] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12779.013692] Lustre: Skipped 1 previous similar message [12779.015222] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12779.042285] Lustre: Skipped 1 previous similar message [12784.098087] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12784.104213] Lustre: Skipped 1 previous similar message [12784.568785] Lustre: server umount lustre-OST0001 complete [12789.227174] LustreError: 330354:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12789.269500] LustreError: 330354:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12793.827055] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12793.829952] LustreError: Skipped 1 previous similar message [12793.844469] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12793.872197] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12799.478198] Lustre: server umount lustre-OST0000 complete [12804.577162] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12804.579344] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12804.580371] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12804.595991] Lustre: Skipped 2 previous similar messages [12804.605293] Lustre: Skipped 2 previous similar messages [12809.598397] Lustre: server umount lustre-MDT0000 complete [12813.770593] LustreError: 332055:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787560961 with bad export cookie 8503494861471124704 [12813.783642] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12813.791649] LustreError: 332055:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12814.279195] Lustre: server umount lustre-MDT0001 complete [12827.836230] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing unload_modules_local [12830.926183] Key type lgssc unregistered [12831.260350] LNet: 333319:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12831.278618] LNetError: 333319:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12831.297706] LNet: Removed LNI 192.168.201.115@tcp [12832.211721] Key type .llcrypt unregistered [12832.213698] Key type ._llcrypt unregistered [12878.019155] Key type ._llcrypt registered [12878.020226] Key type .llcrypt registered [12878.157308] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [12879.732826] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12879.843562] alg: No test for adler32 (adler32-zlib) [12881.106773] Lustre: Lustre: Build Version: 2.17.57_83_g6b3cfd8 [12881.558882] LNet: Added LNI 192.168.201.115@tcp [8/256/0/180] [12883.431313] Key type lgssc registered [12885.784183] Lustre: Echo OBD driver; http://www.lustre.org/ [12897.948018] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12897.981542] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12899.422331] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12904.955393] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [12909.541541] LustreError: 335611:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12909.558235] LustreError: 335611:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12914.554608] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12914.664700] LustreError: 335612:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12915.005461] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12920.077218] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [12923.635592] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [12931.367503] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12931.645380] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12938.506713] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [12939.819580] LustreError: 337099:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12944.880168] LustreError: 337102:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12945.917213] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [12948.534935] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12948.824720] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12954.150880] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [12957.265777] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [12967.305812] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12975.094757] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [12991.968843] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12991.973359] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12991.992958] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12995.045147] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12995.052582] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12995.065837] Lustre: Skipped 2 previous similar messages [12995.091357] Lustre: Skipped 3 previous similar messages [12997.522283] Lustre: server umount lustre-MDT0000 complete [13000.167242] LustreError: 335611:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13000.185580] LustreError: 335611:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [13001.740493] LustreError: 338620:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787561149 with bad export cookie 943665221569695609 [13001.781423] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13002.392253] Lustre: server umount lustre-MDT0001 complete [13016.531874] Lustre: server umount lustre-OST0000 complete [13032.155181] Lustre: server umount lustre-OST0001 complete [13050.704514] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [13062.742200] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13063.197802] LustreError: 340877:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13063.268928] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13067.242383] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [13076.172293] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13076.690641] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13081.497645] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [13084.660744] Lustre: 342017:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [13091.651905] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13098.711026] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [13101.291698] LustreError: 342848:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13101.308744] LustreError: 342848:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [13108.307267] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13108.517054] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [13108.525165] Lustre: Skipped 1 previous similar message [13109.566960] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [13109.588130] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [13115.459867] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [13123.405885] Lustre: DEBUG MARKER: Using TIMEOUT=20 [13126.568553] Lustre: 343888:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [13130.209718] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13130.215581] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13130.229768] Lustre: Skipped 3 previous similar messages [13130.249406] Lustre: Skipped 3 previous similar messages [13135.332341] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13135.343974] Lustre: Skipped 2 previous similar messages [13136.225359] Lustre: server umount lustre-MDT0000 complete [13140.347949] LustreError: 340859:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787561288 with bad export cookie 943665221569697310 [13140.365678] LustreError: 340859:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [13140.380624] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13140.449372] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13140.452537] LustreError: 340874:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13140.469374] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13140.472381] LustreError: 340874:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [13140.478477] Lustre: Skipped 2 previous similar messages [13140.726103] Lustre: server umount lustre-MDT0001 complete [13146.873562] Lustre: server umount lustre-OST0000 complete [13151.113720] Lustre: server umount lustre-OST0001 complete [13155.168486] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [13156.897646] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [13158.867754] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 04:48:25 (1787561305) [13187.956790] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [13196.963767] loop0: detected capacity change from 0 to 1099511627776 [13197.587318] SGI XFS with ACLs, security attributes, quota, no debug enabled [13197.592808] *** VALIDATE xfs *** [13197.604363] XFS (loop0): Mounting V5 Filesystem [13197.635993] XFS (loop0): Ending clean mount [13198.991359] loop1: detected capacity change from 0 to 18691697672192 [13314.947164] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_hostid [13325.398944] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [13366.959852] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13367.349723] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13367.377453] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13367.490115] Lustre: lustre-MDT0000: new disk, initializing [13367.588876] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13367.626100] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13372.003890] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [13384.056948] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13384.172564] Lustre: 349041:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13384.228479] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13384.232090] Lustre: Skipped 1 previous similar message [13384.305645] Lustre: lustre-MDT0001: new disk, initializing [13384.453198] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13384.465955] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13389.970157] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [13400.584476] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13407.564480] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13414.790637] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13415.055164] Lustre: lustre-OST0000: new disk, initializing [13415.059708] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13415.067951] Lustre: 351010:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13416.814476] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13416.828528] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13417.005043] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13422.194814] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [13433.099446] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13437.938506] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [13438.190460] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [13442.307762] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [13442.465271] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [13447.649377] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [13447.656577] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13447.662457] LustreError: Skipped 1 previous similar message [13447.666897] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13447.680420] Lustre: Skipped 2 previous similar messages [13447.693273] Lustre: Skipped 1 previous similar message [13450.679136] Lustre: server umount lustre-OST0000 complete [13456.352453] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13456.355614] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13461.125365] Lustre: server umount lustre-MDT0000 complete [13463.010048] LustreError: 349046:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13463.026148] LustreError: 349046:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [13464.745882] LustreError: 349031:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787561612 with bad export cookie 943665221569698864 [13464.748226] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13464.755050] LustreError: 349031:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [13465.121154] Lustre: server umount lustre-MDT0001 complete [13472.610803] XFS (loop0): Unmounting Filesystem [13476.259640] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 04:53:43 (1787561623) [13483.886358] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13484.718545] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13484.728346] Lustre: Skipped 2 previous similar messages [13490.864239] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [13499.429620] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13504.812991] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [13513.818748] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13520.603183] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13528.538746] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13535.334707] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [13544.078536] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13555.681264] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [13555.692805] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13555.718907] Lustre: Skipped 1 previous similar message [13555.729563] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13555.738397] Lustre: Skipped 3 previous similar messages [13559.786478] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [13561.092342] Lustre: server umount lustre-OST0000 complete [13580.257797] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [13580.470632] Lustre: server umount lustre-MDT0000 complete [13584.404283] LustreError: 353549:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787561732 with bad export cookie 943665221569699746 [13584.419560] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13584.745530] Lustre: server umount lustre-MDT0001 complete [13596.226313] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing unload_modules_local [13600.289351] Key type lgssc unregistered [13600.570915] LNet: 357202:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13600.578536] LNetError: 357202:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13600.597952] LNet: Removed LNI 192.168.201.115@tcp [13601.401154] Key type .llcrypt unregistered [13601.403206] Key type ._llcrypt unregistered [13629.001715] Key type ._llcrypt registered [13629.003921] Key type .llcrypt registered [13629.107610] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [13629.856746] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [13629.969165] alg: No test for adler32 (adler32-zlib) [13631.005748] Lustre: Lustre: Build Version: 2.17.57_83_g6b3cfd8 [13631.206195] LNet: Added LNI 192.168.201.115@tcp [8/256/0/180] [13632.864071] Key type lgssc registered [13633.929743] Lustre: Echo OBD driver; http://www.lustre.org/ [13640.885801] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [13640.901214] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13642.308709] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13646.740125] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [13652.453554] LustreError: 358768:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13652.461911] LustreError: 358768:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [13653.967543] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13654.225201] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13657.650973] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [13664.679629] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13669.853613] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13675.701237] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13676.148615] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13682.357378] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [13689.278401] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13697.422680] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.15@tcp (stopping) [13698.017384] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13698.036114] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13698.051673] Lustre: Skipped 1 previous similar message [13699.058436] Lustre: server umount lustre-MDT0001 complete [13702.548039] LustreError: 361064:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.15@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13707.651428] LustreError: 358764:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.15@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13707.678531] LustreError: 358764:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [13710.681900] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13710.751074] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13710.757917] Lustre: Found index 1 for lustre-MDT0001, updating log [13710.805563] Lustre: 358755:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13711.124943] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13716.487402] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13716.973292] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [13720.079336] LustreError: 362339:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13720.085545] LustreError: 362339:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13720.103857] LustreError: 358760:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13720.264371] LustreError: 362346:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.115@tcp: cfg command failed: rc = -17 [13720.282502] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [13720.282502] [13728.141398] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.15@tcp (stopping) [13731.808090] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13731.813680] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13731.815499] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13731.825137] Lustre: Skipped 2 previous similar messages [13731.832498] Lustre: Skipped 1 previous similar message [13733.905243] Lustre: server umount lustre-MDT0001 complete [13736.929383] LustreError: 359499:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13736.938838] LustreError: 359499:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [13742.059710] LustreError: 358763:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13742.074101] LustreError: 358763:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [13743.341386] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13743.399136] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13743.401632] Lustre: Found index 1 for lustre-MDT0001, updating log [13743.425840] Lustre: 358757:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13743.645243] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13747.447939] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [13748.732074] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13748.766359] LustreError: 363362:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13748.774582] LustreError: 363362:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13748.781239] LustreError: 358760:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13748.788250] LustreError: 358760:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13748.886902] LustreError: 363368:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [13748.896036] LustreError: 363368:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [13748.900986] LustreError: 363368:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.115@tcp: cfg command failed: rc = -17 [13748.911872] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.201.115@tcp [13748.911872] [14055.831361] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.15@tcp (stopping) [14055.833663] Lustre: Skipped 1 previous similar message [14055.909546] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14055.930670] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [14060.706108] Lustre: server umount lustre-MDT0001 complete [14060.920885] LustreError: 358764:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.15@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [14060.948433] LustreError: 358764:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [14071.703355] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14071.769907] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [14071.775270] Lustre: Found index 1 for lustre-MDT0001, updating log [14071.804482] Lustre: 358758:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [14072.065198] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [14076.270456] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [14077.423945] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [14078.481082] LustreError: 371991:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [14078.485852] LustreError: 371991:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [14078.507196] LustreError: 358760:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [14078.516048] LustreError: 358760:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [14078.587132] LustreError: 371999:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [14078.602209] LustreError: 371999:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.115@tcp: cfg command failed: rc = -114 [14078.612183] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [14078.612183] [14399.968181] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [14399.971937] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14399.979856] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14399.979865] Lustre: Skipped 2 previous similar messages [14400.006934] Lustre: Skipped 3 previous similar messages [14403.679528] Lustre: server umount lustre-MDT0000 complete [14405.093218] LustreError: 358765:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14405.119986] LustreError: 358765:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 10 previous similar messages [14407.221189] LustreError: 358748:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787562555 with bad export cookie 15641656891039440475 [14407.227900] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14407.234794] LustreError: 358748:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [14410.211531] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14413.854238] Lustre: server umount lustre-MDT0001 complete [14418.696540] Lustre: server umount lustre-OST0000 complete [14428.926945] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_hostid [14442.061364] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [14483.751249] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 05:10:29 (1787562629) [14490.742851] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14490.959139] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [14490.978091] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [14491.054694] Lustre: lustre-MDT0000: new disk, initializing [14491.140840] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [14491.156783] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [14495.964498] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [14506.688870] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14506.812699] Lustre: 382366:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [14506.871725] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [14506.875285] Lustre: Skipped 1 previous similar message [14506.992254] Lustre: lustre-MDT0001: new disk, initializing [14507.129513] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [14507.193765] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [14507.198580] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [14512.302315] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [14521.693136] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [14527.498835] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [14535.153082] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14535.685559] Lustre: lustre-OST0000: new disk, initializing [14535.689240] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [14535.705890] Lustre: 384333:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [14535.921943] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [14543.517936] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_default_debug -1 all [14544.958639] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [14544.973425] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [14545.072940] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [14553.989296] Lustre: DEBUG MARKER: oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [14562.795110] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [14562.805446] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14562.826605] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14562.838404] Lustre: Skipped 4 previous similar messages [14564.113054] Lustre: server umount lustre-MDT0000 complete [14565.862292] LustreError: 382376:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14565.892942] LustreError: 382376:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [14568.677673] LustreError: 382356:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787562716 with bad export cookie 15641656891039442428 [14568.688716] LustreError: 383171:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [14568.701911] LustreError: MGC192.168.201.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14568.706977] LustreError: 382356:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [14569.193962] Lustre: server umount lustre-MDT0001 complete [14583.386414] Lustre: server umount lustre-OST0000 complete [14612.609556] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing set_hostid [14624.114494] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing load_modules_local [14660.959648] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 14287 sec ========= 05:13:26 (1787562806) [14663.541841] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 05:13:29 (1787562809) === [14668.046896] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 05:13:33 (1787562813) === [14714.955467] Lustre: DEBUG MARKER: oleg115-server.virtnet: executing unload_modules_local [14717.535760] Key type lgssc unregistered [14717.797634] LNet: 390703:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [14717.805247] LNetError: 390703:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [14717.817768] LNet: Removed LNI 192.168.201.115@tcp [14718.653333] Key type .llcrypt unregistered [14718.655851] Key type ._llcrypt unregistered