[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 505167005 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.002374] x2apic enabled [ 0.003009] Switched APIC routing to physical x2apic. [ 0.004015] kvm-guest: setup PV IPIs [ 0.007000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007028] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008014] pid_max: default: 32768 minimum: 301 [ 0.009141] LSM: Security Framework initializing [ 0.010062] Yama: becoming mindful. [ 0.011036] SELinux: Initializing. [ 0.012089] *** VALIDATE selinux *** [ 0.020868] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.025544] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.026168] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.027116] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028134] *** VALIDATE tmpfs *** [ 0.030307] *** VALIDATE proc *** [ 0.031171] *** VALIDATE cgroup *** [ 0.032009] *** VALIDATE cgroup2 *** [ 0.033283] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.034167] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.035009] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.036032] Spectre V2 : User space: Vulnerable [ 0.037010] Speculative Store Bypass: Vulnerable [ 0.039943] debug: unmapping init [mem 0xffffffffa4c59000-0xffffffffa4c60fff] [ 0.042000] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.042646] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.043022] ... version: 2 [ 0.044013] ... bit width: 48 [ 0.045013] ... generic registers: 4 [ 0.046011] ... value mask: 0000ffffffffffff [ 0.047008] ... max period: 00007fffffffffff [ 0.048014] ... fixed-purpose events: 3 [ 0.048876] ... event mask: 000000070000000f [ 0.049338] rcu: Hierarchical SRCU implementation. [ 0.051442] smp: Bringing up secondary CPUs ... [ 0.052712] x86: Booting SMP configuration: [ 0.053034] .... node #0, CPUs: #1 #2 #3 [ 0.057417] smp: Brought up 1 node, 4 CPUs [ 0.059018] smpboot: Max logical packages: 1 [ 0.060019] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.127561] node 0 deferred pages initialised in 65ms [ 0.131117] devtmpfs: initialized [ 0.132203] x86/mm: Memory block size: 128MB [ 0.134350] gcov: version magic: 0x41383552 [ 0.135674] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.139098] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.142433] pinctrl core: initialized pinctrl subsystem [ 0.144206] [ 0.144677] ************************************************************* [ 0.146017] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.149013] ** ** [ 0.151012] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.153014] ** ** [ 0.156013] ** This means that this kernel is built to expose internal ** [ 0.158015] ** IOMMU data structures, which may compromise security on ** [ 0.160015] ** your system. ** [ 0.162013] ** ** [ 0.165015] ** If you see this message and you are not debugging the ** [ 0.167012] ** kernel, report this immediately to your vendor! ** [ 0.169021] ** ** [ 0.171015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.174013] ************************************************************* [ 0.176611] NET: Registered protocol family 16 [ 0.178376] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.180044] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.182062] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.185100] cpuidle: using governor menu [ 0.186457] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.188341] PCI: Using configuration type 1 for base access [ 0.190091] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.195126] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.196018] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.198235] cryptd: max_cpu_qlen set to 1000 [ 0.200018] ACPI: Added _OSI(Module Device) [ 0.200992] ACPI: Added _OSI(Processor Device) [ 0.201011] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.201947] ACPI: Added _OSI(Processor Aggregator Device) [ 0.203721] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.211482] ACPI: Interpreter enabled [ 0.212044] ACPI: PM: (supports S0 S3 S4 S5) [ 0.213000] ACPI: Using IOAPIC for interrupt routing [ 0.214120] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.216280] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.225161] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.228051] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.231027] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.234085] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.240518] acpiphp: Slot [2] registered [ 0.242172] acpiphp: Slot [5] registered [ 0.243151] acpiphp: Slot [6] registered [ 0.245119] acpiphp: Slot [7] registered [ 0.246132] acpiphp: Slot [8] registered [ 0.248128] acpiphp: Slot [9] registered [ 0.250122] acpiphp: Slot [10] registered [ 0.251342] acpiphp: Slot [3] registered [ 0.253146] acpiphp: Slot [4] registered [ 0.254124] acpiphp: Slot [11] registered [ 0.256110] acpiphp: Slot [12] registered [ 0.257109] acpiphp: Slot [13] registered [ 0.259105] acpiphp: Slot [14] registered [ 0.260108] acpiphp: Slot [15] registered [ 0.261114] acpiphp: Slot [16] registered [ 0.263526] acpiphp: Slot [17] registered [ 0.265082] acpiphp: Slot [18] registered [ 0.266130] acpiphp: Slot [19] registered [ 0.268100] acpiphp: Slot [20] registered [ 0.269105] acpiphp: Slot [21] registered [ 0.271113] acpiphp: Slot [22] registered [ 0.272119] acpiphp: Slot [23] registered [ 0.274107] acpiphp: Slot [24] registered [ 0.275116] acpiphp: Slot [25] registered [ 0.276152] acpiphp: Slot [26] registered [ 0.278094] acpiphp: Slot [27] registered [ 0.280112] acpiphp: Slot [28] registered [ 0.281101] acpiphp: Slot [29] registered [ 0.283121] acpiphp: Slot [30] registered [ 0.284123] acpiphp: Slot [31] registered [ 0.286062] PCI host bridge to bus 0000:00 [ 0.287024] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.289029] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.292025] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.294031] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.296021] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.298022] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.300186] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.302757] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.305142] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.313016] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.317101] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.319017] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.320009] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.322016] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.324445] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.326073] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.329059] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.332801] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.338017] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.353020] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.357007] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.362265] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.371017] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.380018] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.392015] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.402711] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.409016] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.417018] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.432021] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.442931] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.450021] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.457022] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.470017] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.477340] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.484021] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.493018] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.509020] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.522753] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.529021] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.538016] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.554017] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.568869] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.574018] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.581023] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.602022] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.613550] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.615427] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.618372] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.620417] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.623249] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.628148] iommu: Default domain type: Passthrough [ 0.630494] SCSI subsystem initialized [ 0.632172] ACPI: bus type USB registered [ 0.634119] usbcore: registered new interface driver usbfs [ 0.636094] usbcore: registered new interface driver hub [ 0.638103] usbcore: registered new device driver usb [ 0.639141] pps_core: LinuxPPS API ver. 1 registered [ 0.641011] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.644054] PTP clock support registered [ 0.646067] EDAC MC: Ver: 3.0.0 [ 0.648150] PCI: Using ACPI for IRQ routing [ 0.650897] NetLabel: Initializing [ 0.652012] NetLabel: domain hash size = 128 [ 0.654012] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.656081] NetLabel: unlabeled traffic allowed by default [ 0.658174] vgaarb: loaded [ 0.659194] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.660018] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.669225] clocksource: Switched to clocksource kvm-clock [ 0.789640] VFS: Disk quotas dquot_6.6.0 [ 0.791459] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.794257] *** VALIDATE ramfs *** [ 0.795513] *** VALIDATE hugetlbfs *** [ 0.797245] pnp: PnP ACPI init [ 0.799461] pnp: PnP ACPI: found 6 devices [ 0.825960] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.829017] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.831015] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.833419] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.836500] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.838922] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.841851] NET: Registered protocol family 2 [ 0.844477] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.849275] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.852711] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.858288] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.861705] TCP: Hash tables configured (established 65536 bind 65536) [ 0.864922] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.868148] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.871324] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.873700] NET: Registered protocol family 1 [ 0.876672] RPC: Registered named UNIX socket transport module. [ 0.878442] RPC: Registered udp transport module. [ 0.879822] RPC: Registered tcp transport module. [ 0.881123] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.883309] NET: Registered protocol family 44 [ 0.884700] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.887066] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.889414] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.892230] PCI: CLS 0 bytes, default 64 [ 0.894067] Unpacking initramfs... [ 2.270785] debug: unmapping init [mem 0xffff8e04bcc54000-0xffff8e04bffbffff] [ 2.273904] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.276392] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.279390] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.760358] Initialise system trusted keyrings [ 2.762025] Key type blacklist registered [ 2.764618] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.773636] zbud: loaded [ 2.776552] *** VALIDATE nfs *** [ 2.777915] *** VALIDATE nfs4 *** [ 2.779231] pstore: using deflate compression [ 2.782600] Platform Keyring initialized [ 2.877493] NET: Registered protocol family 38 [ 2.879400] Key type asymmetric registered [ 2.881057] Asymmetric key parser 'x509' registered [ 2.883030] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.886307] io scheduler mq-deadline registered [ 2.887941] io scheduler kyber registered [ 2.889684] io scheduler bfq registered [ 2.891497] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.894333] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.897288] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.900110] ACPI: Power Button [PWRF] [ 2.904959] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.911517] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.928033] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 2.937780] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 2.967030] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.995564] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.024213] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.029068] Non-volatile memory driver v1.3 [ 3.030650] Linux agpgart interface v0.103 [ 3.060606] virtio_blk virtio1: [vda] 146728 512-byte logical blocks (75.1 MB/71.6 MiB) [ 3.063685] vda: detected capacity change from 0 to 75124736 [ 3.082661] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.085670] vdb: detected capacity change from 0 to 1073741824 [ 3.108269] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.111970] vdc: detected capacity change from 0 to 2621440000 [ 3.133488] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.136789] vdd: detected capacity change from 0 to 2621440000 [ 3.156749] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.159917] vde: detected capacity change from 0 to 4294967296 [ 3.174236] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.177492] vdf: detected capacity change from 0 to 4294967296 [ 3.184346] libphy: Fixed MDIO Bus: probed [ 3.204134] usbcore: registered new interface driver usbserial_generic [ 3.205596] usbserial: USB Serial support registered for generic [ 3.207152] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.211630] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.213846] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.215983] mousedev: PS/2 mouse device common for all mice [ 3.220595] rtc_cmos 00:05: RTC can wake from S4 [ 3.224476] rtc_cmos 00:05: registered as rtc0 [ 3.226245] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.229455] intel_pstate: CPU model not supported [ 3.232722] hid: raw HID events driver (C) Jiri Kosina [ 3.234698] usbcore: registered new interface driver usbhid [ 3.236786] usbhid: USB HID core driver [ 3.237080] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.238222] drop_monitor: Initializing network drop monitor service [ 3.244763] Initializing XFRM netlink socket [ 3.246772] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.247314] NET: Registered protocol family 10 [ 3.252739] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.253038] Segment Routing with IPv6 [ 3.257318] NET: Registered protocol family 17 [ 3.259081] mpls_gso: MPLS GSO support [ 3.266454] RAS: Correctable Errors collector initialized. [ 3.268584] AVX version of gcm_enc/dec engaged. [ 3.270259] AES CTR mode by8 optimization enabled [ 3.340766] sched_clock: Marking stable (3340745189, 0)->(4220474119, -879728930) [ 3.344193] registered taskstats version 1 [ 3.346148] Loading compiled-in X.509 certificates [ 3.348183] zswap: loaded using pool lzo/zbud [ 3.372931] Key type big_key registered [ 3.385767] Key type encrypted registered [ 3.387410] ima: No TPM chip found, activating TPM-bypass! [ 3.389243] ima: Allocated hash algorithm: sha1 [ 3.390676] ima: No architecture policies found [ 3.392339] evm: Initialising EVM extended attributes: [ 3.394016] evm: security.selinux [ 3.395251] evm: security.ima [ 3.396372] evm: security.capability [ 3.397541] evm: HMAC attrs: 0x1 [ 3.399768] rtc_cmos 00:05: setting system clock to 2026-09-08 03:05:12 UTC (1788836712) [ 3.405906] debug: unmapping init [mem 0xffffffffa5c03000-0xffffffffa5dfffff] [ 3.408855] debug: unmapping init [mem 0xffffffffa4982000-0xffffffffa4c58fff] [ 3.418120] Write protecting the kernel read-only data: 28672k [ 3.421264] debug: unmapping init [mem 0xffffffffa3003000-0xffffffffa31fffff] [ 3.424040] debug: unmapping init [mem 0xffffffffa3914000-0xffffffffa39fffff] [ 3.456930] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.465489] systemd[1]: Detected virtualization kvm. [ 3.467598] systemd[1]: Detected architecture x86-64. [ 3.469588] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.495163] systemd[1]: No hostname configured. [ 3.497019] systemd[1]: Set hostname to . [ 3.499145] random: systemd: uninitialized urandom read (16 bytes read) [ 3.501669] systemd[1]: Initializing machine ID from random generator. [ 3.540856] random: ln: uninitialized urandom read (6 bytes read) [ 3.616419] random: systemd: uninitialized urandom read (16 bytes read) [ 3.619472] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 3.624063] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ 3.628353] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ OK ] Reached target Initrd Root Device. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on Journal Socket. Starting Setup Virtual Console... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Slices. [ OK ] Reached target Paths. Starting Apply Kernel Variables... [ OK ] Reached target Timers. Starting Create Volatile Files and Directories... [ OK ] Started Memstrack Anylazing Service. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Sockets. Starting Journal Service... [ OK ] Reached target Swap. [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.155940] device-mapper: uevent: version 1.0.3 [ 4.157411] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 4.812039] random: fast init done [ 4.814923] virtio_net virtio0 ens2: renamed from eth0 [ 4.836556] scsi host0: ata_piix [ 4.859187] scsi host1: ata_piix [ 4.861022] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 4.863236] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 8.521252] dracut-initqueue[578]: RTNETLINK answers: File exists [ 9.903955] random: crng init done [ 9.905444] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.198507] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Timers. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Swap. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.260324] printk: systemd: 26 output lines suppressed due to ratelimiting [ 11.478968] SELinux: Disabled at runtime. [ 11.534054] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.542731] systemd[1]: Detected virtualization kvm. [ 11.544656] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 11.993290] systemd[1]: initrd-switch-root.service: Succeeded. [ 11.996492] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.000541] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.004040] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.007231] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.012286] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.019866] systemd[1]: Mounting Kernel Debug File System... Mounting Kernel Debug File System... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Created slice system-getty.slice. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Listening on Process Core Dump Socket. Activating swap /dev/disk/by-label/SWAP... Mounting Huge Pages File System... [ OK ] Listening on udev Kernel Socket. Mounting POSIX Message Queue File System... [ 12.105843] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. Starting Remount Root and Kernel File Systems... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice User and Session Slice. [ OK ] Listening on initctl Compatibility Named Pipe. Starting Apply Kernel Variables... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Reached target rpc_pipefs.target. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Stopped target Initrd File Systems. [ OK ] Reached target Slices. [ OK ] Started Journal Service. [ OK ] Mounted Kernel Debug File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted POSIX Message Queue File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 12.367820] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Coldplug all Devices. [ OK ] Started udev Kernel Device Manager. [ 12.640646] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 12.646211] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 12.768205] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 12.782366] EDAC sbridge: Ver: 1.1.2 [ 14.039020] Key type dns_resolver registered [ 14.308262] NFS: Registering the id_resolver key type [ 14.311174] Key type id_resolver registered [ 14.312526] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Mark the need to relabel after reboot... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started irqbalance daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Reached target sshd-keygen.target. Starting Login Service... [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg119-server login: [ 39.255510] libcfs: loading out-of-tree module taints kernel. [ 39.271036] Key type ._llcrypt registered [ 39.271993] Key type .llcrypt registered [ 39.312648] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_hostid [ 54.520788] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 56.490907] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 56.501667] alg: No test for adler32 (adler32-zlib) [ 58.125285] Lustre: Lustre: Build Version: 2.17.57_113_gf5ca795 [ 59.265766] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 61.073275] Key type lgssc registered [ 61.176774] hrtimer: interrupt took 23867934 ns [ 63.214308] Lustre: Echo OBD driver; http://www.lustre.org/ [ 85.865394] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 135.638501] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 151.320506] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 151.364911] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 152.681399] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 152.711160] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 152.779948] Lustre: lustre-MDT0000: new disk, initializing [ 152.840527] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 152.854257] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 157.676973] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 172.233279] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 172.372888] Lustre: 6502:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 172.447894] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 172.452578] Lustre: Skipped 1 previous similar message [ 172.588949] Lustre: lustre-MDT0001: new disk, initializing [ 172.713681] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 172.769341] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 172.794736] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 178.491935] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 183.899357] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 194.331083] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 194.688322] Lustre: lustre-OST0000: new disk, initializing [ 194.693568] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 194.699492] Lustre: 8441:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 194.784413] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 200.920680] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 202.792655] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 202.803687] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 202.843262] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 215.543513] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 215.663608] Lustre: lustre-OST0001: new disk, initializing [ 215.668557] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 215.673790] Lustre: 9514:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 215.751304] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 222.490911] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 222.801748] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 222.811071] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 222.906943] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 234.714547] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 238.849890] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 246.355250] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing check_logdir /tmp/testlogs/ [ 251.542390] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing yml_node [ 256.284359] Lustre: DEBUG MARKER: Client: 2.17.57.113 [ 259.008768] Lustre: DEBUG MARKER: MDS: 2.17.57.113 [ 262.077090] Lustre: DEBUG MARKER: OSS: 2.17.57.113 [ 264.347920] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Mon Sep 7 23:09:31 EDT 2026 [ 285.201752] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 286.863019] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 305.142380] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 305.154965] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 305.182669] Lustre: Skipped 1 previous similar message [ 305.202716] Lustre: Skipped 3 previous similar messages [ 309.432934] Lustre: server umount lustre-MDT0000 complete [ 314.003937] LustreError: 6493:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788837023 with bad export cookie 4108910481700944383 [ 314.007073] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 314.011145] LustreError: 6493:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 315.361079] LustreError: 10090:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 315.364230] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 315.370611] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 315.386464] LustreError: 10090:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 315.417766] Lustre: Skipped 3 previous similar messages [ 319.402220] LustreError: 11515:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 319.407277] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 319.431927] LustreError: 11515:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 319.447334] Lustre: Skipped 2 previous similar messages [ 320.571205] Lustre: server umount lustre-MDT0001 complete [ 325.665351] Lustre: server umount lustre-OST0000 complete [ 330.246357] Lustre: server umount lustre-OST0001 complete [ 338.784549] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_hostid [ 349.527389] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 389.943954] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 390.260914] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 390.285943] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 390.354762] Lustre: lustre-MDT0000: new disk, initializing [ 390.546498] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 390.575231] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 396.269448] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 409.205624] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 409.360636] Lustre: 15070:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 409.443306] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 409.448273] Lustre: Skipped 1 previous similar message [ 409.540115] Lustre: lustre-MDT0001: new disk, initializing [ 409.672438] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 409.711778] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 409.745388] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 415.790322] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 427.273623] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 434.328858] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 442.419315] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 442.897303] Lustre: lustre-OST0000: new disk, initializing [ 442.900727] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 442.908697] Lustre: 17042:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 443.049873] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 444.745303] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 444.772536] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 444.947565] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 451.948762] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 463.255956] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 469.609331] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 469.827558] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 475.203825] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 475.380128] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 480.739591] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 480.754145] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 480.758580] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 480.765184] Lustre: Skipped 1 previous similar message [ 480.782541] Lustre: Skipped 1 previous similar message [ 483.695403] Lustre: server umount lustre-OST0000 complete [ 491.000617] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 491.012692] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 491.019856] Lustre: Skipped 1 previous similar message [ 491.029187] Lustre: Skipped 1 previous similar message [ 494.516191] Lustre: server umount lustre-MDT0000 complete [ 496.099553] LustreError: 16986:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 496.124299] LustreError: 16986:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 498.988918] LustreError: 15867:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788837208 with bad export cookie 4108910481700945916 [ 498.991170] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 499.007533] LustreError: 15867:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 499.065808] LustreError: 3632:0:(import.c:717:ptlrpc_connect_import_locked()) already connecting [ 499.626829] Lustre: server umount lustre-MDT0001 complete [ 505.666574] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 23:13:33 (1788837213) [ 512.980453] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 513.493919] LustreError: 19308:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 513.622424] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 518.629221] LustreError: 19309:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 518.871583] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 528.881041] LustreError: 19308:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 528.899304] LustreError: 19308:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 529.288799] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 535.892118] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 545.439596] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 553.888356] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 562.842977] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 571.372364] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 580.843309] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 590.346944] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 598.499983] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 598.503669] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 598.526354] Lustre: Skipped 2 previous similar messages [ 598.529176] Lustre: Skipped 1 previous similar message [ 604.863066] Lustre: server umount lustre-MDT0000 complete [ 608.758808] LustreError: 19308:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 608.797334] LustreError: 19308:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 610.530731] LustreError: 19289:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788837319 with bad export cookie 4108910481700946798 [ 610.566556] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 611.717630] Lustre: server umount lustre-MDT0001 complete [ 627.960251] Lustre: server umount lustre-OST0000 complete [ 650.364570] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 662.280413] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 662.964427] LustreError: 23690:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 663.050532] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 663.055540] Lustre: Skipped 2 previous similar messages [ 668.585121] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 680.110233] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 686.864628] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 690.988597] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 700.983433] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 709.067445] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 719.788055] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 720.018511] Lustre: lustre-OST0001: new disk, initializing [ 720.032945] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 720.042491] Lustre: 26009:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 721.276741] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 721.286862] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 721.479056] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 728.402000] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 739.891161] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 744.501340] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 762.347631] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 762.356889] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 762.374834] Lustre: Skipped 3 previous similar messages [ 762.394863] Lustre: Skipped 7 previous similar messages [ 764.793620] Lustre: server umount lustre-MDT0000 complete [ 767.460771] LustreError: 26290:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 767.488526] LustreError: 26290:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 769.708596] LustreError: 26008:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788837478 with bad export cookie 4108910481700947645 [ 769.726764] LustreError: 26008:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 769.727165] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 770.389069] Lustre: server umount lustre-MDT0001 complete [ 785.907598] Lustre: server umount lustre-OST0000 complete [ 788.640774] Lustre: 3635:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788837481/real 1788837481] req@ffff8e0407769f80 x1875731303101056/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788837497 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 788.670098] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 790.089803] Lustre: server umount lustre-OST0001 complete [ 810.536655] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 821.778198] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 822.314405] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 822.319189] Lustre: Skipped 3 previous similar messages [ 826.674366] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 836.380578] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 841.412862] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 844.564755] Lustre: 30174:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 852.082729] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 860.433793] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 871.157705] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 878.927338] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 887.607417] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 891.731112] Lustre: 32052:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 902.139650] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 902.151336] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 902.164034] Lustre: Skipped 2 previous similar messages [ 902.179567] Lustre: Skipped 3 previous similar messages [ 905.368033] Lustre: server umount lustre-MDT0000 complete [ 907.238470] LustreError: 29027:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 907.257614] LustreError: 29027:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 909.193701] LustreError: 29010:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788837618 with bad export cookie 4108910481700948940 [ 909.196488] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 909.202522] LustreError: 29010:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 909.458335] Lustre: server umount lustre-MDT0001 complete [ 924.366240] Lustre: server umount lustre-OST0000 complete [ 938.731407] Lustre: server umount lustre-OST0001 complete [ 948.604690] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 23:20:55 (1788837655) [ 976.969898] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 986.705990] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 991.509168] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 1000.091859] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1006.840452] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 1011.267842] Lustre: 36011:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1020.061584] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1027.634620] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 1036.553968] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1043.641259] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 1052.183486] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1056.553564] Lustre: 37889:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1062.376487] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1062.381865] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1062.392442] Lustre: Skipped 2 previous similar messages [ 1062.400697] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1062.406319] Lustre: Skipped 3 previous similar messages [ 1062.414580] LustreError: Skipped 1 previous similar message [ 1068.805843] Lustre: server umount lustre-MDT0000 complete [ 1072.521809] LustreError: 34841:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788837781 with bad export cookie 4108910481700950046 [ 1072.544309] LustreError: 34841:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1072.549253] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1072.792091] Lustre: server umount lustre-MDT0001 complete [ 1079.054296] Lustre: server umount lustre-OST0000 complete [ 1082.623466] Lustre: server umount lustre-OST0001 complete [ 1091.657249] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 23:23:18 (1788837798) [ 1108.890523] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 1120.838496] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1121.566499] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1121.577981] Lustre: Skipped 7 previous similar messages [ 1127.080419] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 1137.081142] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1142.160529] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 1145.287626] Lustre: 41176:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1153.578154] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1161.282702] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 1166.509576] LustreError: 41533:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1166.532582] LustreError: 41533:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 18 previous similar messages [ 1171.723112] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1180.063406] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 1189.402211] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1193.885572] Lustre: 43057:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1194.937354] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1194.942185] Lustre: Skipped 1 previous similar message [ 1198.605923] Lustre: lustre-MDT0000: Not available for connect from 192.168.201.19@tcp (stopping) [ 1198.618498] Lustre: Skipped 6 previous similar messages [ 1199.074559] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1199.084706] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1199.115918] Lustre: Skipped 4 previous similar messages [ 1204.912757] Lustre: server umount lustre-MDT0000 complete [ 1209.505372] LustreError: 40009:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788837918 with bad export cookie 4108910481700951145 [ 1209.505930] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1209.511942] LustreError: 40009:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1215.898799] Lustre: server umount lustre-MDT0001 complete [ 1225.909497] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1235.873935] LustreError: 3631:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff8e0540458000 x1875731303259648/t0(0) o250->MGC192.168.201.119@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1241.949738] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 1244.537859] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 1252.101962] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1257.418071] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 1257.965019] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1257.969417] Lustre: Skipped 1 previous similar message [ 1265.305185] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1266.910494] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1273.596409] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1275.641489] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1283.041725] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1287.748675] Lustre: server umount lustre-MDT0000 complete [ 1291.966856] LustreError: 40009:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788838001 with bad export cookie 4108910481700952279 [ 1291.976995] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1291.979642] LustreError: 40009:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1292.299709] Lustre: server umount lustre-MDT0001 complete [ 1306.760661] Lustre: server umount lustre-OST0000 complete [ 1309.729124] Lustre: 3632:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788838002/real 1788838002] req@ffff8e053625ea00 x1875731303295872/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788838018 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1311.613650] Lustre: server umount lustre-OST0001 complete [ 1321.595134] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 23:27:08 (1788838028) [ 1337.225650] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 1347.743926] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1352.886425] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 1363.118058] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1369.124808] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 1372.819305] Lustre: 48818:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1380.450332] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1387.961697] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 1397.868528] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1404.744314] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 1413.022753] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1417.900800] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1429.991313] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1430.125297] Lustre: server umount lustre-MDT0000 complete [ 1434.025463] LustreError: 47648:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788838143 with bad export cookie 4108910481700953077 [ 1434.039147] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1434.330475] Lustre: server umount lustre-MDT0001 complete [ 1441.103689] Lustre: server umount lustre-OST0000 complete [ 1445.060956] Lustre: server umount lustre-OST0001 complete [ 1456.301271] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 23:29:22 (1788838162) [ 1459.806765] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1462.832353] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 23:29:29 (1788838169) [ 1476.832207] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_hostid [ 1487.588810] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 1512.427752] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1512.818394] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1512.914535] Lustre: lustre-MDT0000: new disk, initializing [ 1513.004368] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1518.384654] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 1528.805531] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1536.772750] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1537.131595] Lustre: lustre-OST0000: new disk, initializing [ 1537.142880] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1537.148613] Lustre: Skipped 1 previous similar message [ 1537.165641] Lustre: 54894:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1539.231520] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1539.253392] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1539.302627] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1544.341563] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 1555.355856] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1558.413797] Lustre: DEBUG MARKER: create test files [ 1600.997642] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1600.998811] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1601.029761] Lustre: Skipped 15 previous similar messages [ 1601.031738] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1601.031745] Lustre: Skipped 18 previous similar messages [ 1603.378600] Lustre: server umount lustre-OST0000 complete [ 1607.371169] Lustre: server umount lustre-MDT0000 complete [ 1619.329169] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 1622.650633] Key type lgssc unregistered [ 1622.990253] LNet: 56526:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1622.996928] LNetError: 56526:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1623.027122] LNet: Removed LNI 192.168.201.119@tcp [ 1624.249341] Key type .llcrypt unregistered [ 1624.252060] Key type ._llcrypt unregistered [ 1655.183157] Key type ._llcrypt registered [ 1655.186922] Key type .llcrypt registered [ 1655.324937] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 1656.865337] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1656.922267] alg: No test for adler32 (adler32-zlib) [ 1658.037105] Lustre: Lustre: Build Version: 2.17.57_113_gf5ca795 [ 1658.311943] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 1660.000220] Key type lgssc registered [ 1661.554750] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1670.867450] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1670.882990] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1672.299810] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1677.930329] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 1686.501601] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1693.263527] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1693.676452] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1697.796810] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1697.931179] Lustre: 59262:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1697.951222] Lustre: 59262:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1697.957107] Lustre: 59262:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1697.969681] Lustre: 59262:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1697.983809] Lustre: 59262:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1697.999931] Lustre: 59262:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1701.744637] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 1712.085034] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1718.502563] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1725.175258] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1738.735692] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1738.739740] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1738.772296] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1740.043625] Lustre: server umount lustre-OST0000 complete [ 1744.312091] Lustre: server umount lustre-MDT0000 complete [ 1755.313750] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 1758.291710] Key type lgssc unregistered [ 1758.576949] LNet: 60871:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1758.585627] LNetError: 60871:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1759.654763] LNet: Removed LNI 192.168.201.119@tcp [ 1760.625457] Key type .llcrypt unregistered [ 1760.628817] Key type ._llcrypt unregistered [ 1790.029944] Key type ._llcrypt registered [ 1790.032901] Key type .llcrypt registered [ 1790.126848] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 1791.366382] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1791.383152] alg: No test for adler32 (adler32-zlib) [ 1792.401153] Lustre: Lustre: Build Version: 2.17.57_113_gf5ca795 [ 1792.657341] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 1794.344383] Key type lgssc registered [ 1796.029645] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1803.787483] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1803.829267] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1805.431925] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1811.959989] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 1820.778219] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1828.566078] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1828.846181] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1830.895553] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1830.985744] Lustre: 63681:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1830.991810] Lustre: 63681:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1830.996932] Lustre: 63681:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1831.001380] Lustre: 63681:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1831.006929] Lustre: 63681:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1831.021489] Lustre: 63681:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1834.420152] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 1844.082498] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1850.727231] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1861.606614] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1861.614388] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1861.636574] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1866.724375] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1866.931433] Lustre: server umount lustre-OST0000 complete [ 1871.004711] Lustre: server umount lustre-MDT0000 complete [ 1883.993793] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 1887.012908] Key type lgssc unregistered [ 1887.388698] LNet: 65241:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1887.393519] LNetError: 65241:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1888.430300] LNet: Removed LNI 192.168.201.119@tcp [ 1889.468817] Key type .llcrypt unregistered [ 1889.470981] Key type ._llcrypt unregistered [ 1910.126168] Key type ._llcrypt registered [ 1910.130968] Key type .llcrypt registered [ 1910.264451] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_hostid [ 1928.764858] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 1930.523607] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1930.626054] alg: No test for adler32 (adler32-zlib) [ 1931.758276] Lustre: Lustre: Build Version: 2.17.57_113_gf5ca795 [ 1932.119455] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 1933.873547] Key type lgssc registered [ 1935.135962] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1972.529554] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1972.563676] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1973.808446] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1973.847947] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1973.957865] Lustre: lustre-MDT0000: new disk, initializing [ 1974.067757] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1974.082652] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1979.253570] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 1989.044416] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1989.180889] Lustre: 68394:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1989.239896] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1989.243972] Lustre: Skipped 1 previous similar message [ 1989.305300] Lustre: lustre-MDT0001: new disk, initializing [ 1989.408082] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1989.452023] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1989.465242] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1994.482573] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2004.017704] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2010.228517] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2018.263229] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2018.456487] Lustre: lustre-OST0000: new disk, initializing [ 2018.458679] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2018.462824] Lustre: 70363:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2018.503238] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2023.864718] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2026.001024] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2026.008802] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2026.166658] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2034.105402] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2039.432876] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2039.629405] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2045.078926] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2045.427326] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2051.554943] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2051.577148] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2051.613560] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2052.079201] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2052.081675] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2057.189225] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2057.193385] Lustre: Skipped 3 previous similar messages [ 2062.305487] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2062.309578] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2062.322204] Lustre: Skipped 1 previous similar message [ 2062.524866] Lustre: server umount lustre-OST0000 complete [ 2067.430877] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2067.443596] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2067.452799] Lustre: Skipped 1 previous similar message [ 2067.466393] Lustre: Skipped 1 previous similar message [ 2072.546030] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2072.564787] Lustre: Skipped 1 previous similar message [ 2073.097494] Lustre: server umount lustre-MDT0000 complete [ 2077.311338] LustreError: 68386:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788838786 with bad export cookie 10267178488232883456 [ 2077.319667] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2077.330602] LustreError: 68386:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2077.711211] Lustre: server umount lustre-MDT0001 complete [ 2088.316552] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 23:39:55 (1788838795) [ 2102.693453] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 2147.782230] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_hostid [ 2159.117522] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 2199.714310] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2199.918485] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2199.954293] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2200.079929] Lustre: lustre-MDT0000: new disk, initializing [ 2200.162621] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2200.180218] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2205.730404] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2217.839038] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2217.928634] Lustre: 76398:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2217.970789] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2217.974468] Lustre: Skipped 1 previous similar message [ 2218.030613] Lustre: lustre-MDT0001: new disk, initializing [ 2218.200835] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2218.248137] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2218.259761] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2223.185615] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2234.479924] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2240.283489] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2247.744322] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2248.174737] Lustre: lustre-OST0000: new disk, initializing [ 2248.178315] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2248.189027] Lustre: 78366:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2248.326487] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2254.640823] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2255.410096] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2255.417982] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2255.503329] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2263.886929] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2268.248065] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2268.459994] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2272.585064] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2272.773853] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2275.811973] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2275.822746] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2275.846050] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2280.879580] Lustre: server umount lustre-OST0000 complete [ 2285.026963] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2285.038303] LustreError: Skipped 1 previous similar message [ 2285.049366] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2285.061213] Lustre: Skipped 1 previous similar message [ 2290.710642] Lustre: server umount lustre-MDT0000 complete [ 2294.296596] LustreError: 76391:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788839003 with bad export cookie 10267178488232884338 [ 2294.306365] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2294.320676] LustreError: 76391:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2294.716369] Lustre: server umount lustre-MDT0001 complete [ 2304.331203] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 23:43:31 (1788839011) [ 2311.722517] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2312.071666] LustreError: 80782:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2312.102969] LustreError: 80782:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2312.169930] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2317.017396] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2317.284171] LustreError: 80783:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2322.407040] LustreError: 80782:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2327.522872] LustreError: 80783:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2327.548593] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2333.065929] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2341.328175] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2347.921924] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2356.023950] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2356.414969] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2356.422215] Lustre: Skipped 1 previous similar message [ 2362.614854] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2370.576037] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2374.920973] Lustre: 83455:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting ea9bfd10-0a41-4ccc-aadd-b996bc001c2a at adminstrative request [ 2379.244168] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2379.283478] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2389.480055] Lustre: MGS: Client ea9bfd10-0a41-4ccc-aadd-b996bc001c2a (at 0@lo) reconnecting [ 2389.488432] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x8e7c586feca73be4 to 0x8e7c586feca73e84 [ 2389.495950] Lustre: MGC192.168.201.119@tcp: Connection restored to 0@lo (at 0@lo) [ 2413.291626] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2413.582306] Lustre: lustre-OST0001: new disk, initializing [ 2413.588254] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2413.594944] Lustre: 83850:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2420.430627] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2420.453972] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2420.642430] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2421.549785] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2432.705079] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2441.185876] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2441.195395] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2441.208880] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2441.210915] Lustre: Skipped 2 previous similar messages [ 2441.218238] Lustre: Skipped 6 previous similar messages [ 2442.348879] Lustre: server umount lustre-OST0001 complete [ 2446.332982] LustreError: 82606:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2446.358566] LustreError: 82606:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2449.892924] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2449.906123] LustreError: Skipped 1 previous similar message [ 2452.702408] Lustre: server umount lustre-OST0000 complete [ 2461.154331] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2461.163564] Lustre: Skipped 2 previous similar messages [ 2463.282121] Lustre: server umount lustre-MDT0000 complete [ 2466.785144] LustreError: 81511:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2466.802149] LustreError: 81511:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2466.958539] LustreError: 80762:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788839176 with bad export cookie 10267178488232885892 [ 2466.965847] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2466.971297] LustreError: 80762:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2467.461795] Lustre: server umount lustre-MDT0001 complete [ 2476.112762] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 23:46:23 (1788839183) [ 2477.902968] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2479.755930] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 23:46:27 (1788839187) [ 2481.628918] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2483.600345] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 23:46:31 (1788839191) [ 2485.570964] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2488.106121] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 23:46:35 (1788839195) [ 2490.231586] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2492.283458] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 23:46:39 (1788839199) [ 2494.120543] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2500.571592] loop: module loaded [ 2500.593475] loop0: detected capacity change from 0 to 4294967296 [ 2500.660143] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2502.253955] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2505.692601] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2509.032305] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2536.420286] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2536.498605] LustreError: 87970:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2536.504378] LustreError: 87970:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2536.509981] LustreError: 87970:0:(tgt_mount.c:2582:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2536.515618] LustreError: 87970:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2556.507852] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_hostid [ 2566.542730] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 2613.386623] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2613.661163] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2613.685855] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2613.756949] Lustre: lustre-MDT0000: new disk, initializing [ 2613.796112] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2613.801644] Lustre: Skipped 1 previous similar message [ 2613.823454] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2620.739795] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2631.821921] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2631.997921] Lustre: 90839:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2632.202970] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2637.535474] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2648.988198] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2655.053878] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2662.414982] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2662.715479] Lustre: 92813:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2669.170440] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2671.182371] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2671.272695] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2680.495313] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2686.236192] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2686.413136] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2692.326664] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2692.635154] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2697.186740] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2697.187782] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2697.201627] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2697.201638] Lustre: Skipped 6 previous similar messages [ 2697.221695] Lustre: Skipped 2 previous similar messages [ 2697.255302] LustreError: Skipped 2 previous similar messages [ 2701.720366] Lustre: server umount lustre-OST0000 complete [ 2721.248522] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2721.375739] Lustre: server umount lustre-MDT0000 complete [ 2722.787768] LustreError: 91650:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2725.564416] LustreError: 90831:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788839434 with bad export cookie 10267178488232886508 [ 2725.572122] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2725.576067] LustreError: 90831:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2725.872718] Lustre: server umount lustre-MDT0001 complete [ 2735.325462] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 23:50:42 (1788839442) [ 2742.644332] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2743.316044] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 2743.320655] Lustre: Skipped 2 previous similar messages [ 2747.493464] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2758.163741] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2763.265179] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2771.647121] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2778.542785] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2788.782390] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2796.187271] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2804.258372] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2809.496292] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2814.138883] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 4 sec [ 2820.634957] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2821.182877] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2829.905710] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2830.063378] Lustre: lustre-OST0001: new disk, initializing [ 2830.066207] Lustre: Skipped 2 previous similar messages [ 2830.071264] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2830.075206] Lustre: Skipped 3 previous similar messages [ 2830.078950] Lustre: 98627:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2837.754129] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2838.226834] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2838.249216] Lustre: Skipped 2 previous similar messages [ 2838.268512] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2838.380996] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2850.321774] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2856.481380] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2856.829718] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2862.446419] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2862.671327] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2884.869864] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2916.814409] Lustre: Failing over lustre-MDT0000 [ 2917.352614] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2917.355439] LustreError: Skipped 1 previous similar message [ 2917.365782] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2917.388241] Lustre: Skipped 2 previous similar messages [ 2917.395186] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2917.407636] Lustre: Skipped 7 previous similar messages [ 2917.649454] Lustre: server umount lustre-MDT0000 complete [ 2920.422169] LustreError: 95227:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2920.445866] LustreError: 95227:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 2936.804952] Lustre: 66615:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788839629/real 1788839629] req@00000000c241318e x1875733267112448/t0(0) o400->MGC192.168.201.119@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1788839645 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2936.832983] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2945.173928] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2945.178679] LDISKFS-fs (dm-0): recovery complete [ 2945.194334] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2946.017499] LustreError: 66612:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000c528c1be x1875733267120384/t0(0) o250->MGC192.168.201.119@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2946.449560] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2947.065492] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2947.068441] Lustre: 97975:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2951.673190] Lustre: 95227:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2951.678818] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2951.682964] Lustre: 95227:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 2951.826632] LustreError: 101476:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2951.944165] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 2952.170984] LustreError: 101476:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2952.514674] LustreError: 101476:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2952.530422] LustreError: 101476:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2952.857075] LustreError: 101476:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2952.871062] LustreError: 101476:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2952.890191] Lustre: 101476:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2952.914538] Lustre: 101476:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 2953.577577] LustreError: 101476:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2953.594992] LustreError: 101476:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2953.920131] LustreError: 101476:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2953.937948] LustreError: 101476:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2954.932753] Lustre: 101476:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2954.954756] Lustre: 101476:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 5 previous similar messages [ 2955.602842] LustreError: 101476:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2955.637953] LustreError: 101476:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 5 previous similar messages [ 2955.960203] LustreError: 101476:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2955.967627] LustreError: 101476:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 5 previous similar messages [ 2959.161152] Lustre: 101476:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2959.182709] Lustre: 101476:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 2959.836619] LustreError: 101476:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2959.854788] LustreError: 101476:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2960.176212] LustreError: 101476:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2960.193419] LustreError: 101476:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2967.269621] Lustre: 101476:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2967.282904] Lustre: 101476:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 24 previous similar messages [ 2967.931335] LustreError: 101476:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2967.956614] LustreError: 101476:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2968.280589] LustreError: 101476:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2968.296355] LustreError: 101476:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2983.565409] Lustre: 101476:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2983.584960] Lustre: 101476:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 50 previous similar messages [ 2984.220777] LustreError: 101476:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2984.227989] LustreError: 101476:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2984.544466] LustreError: 101476:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2984.551462] LustreError: 101476:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 3007.216243] Lustre: 101476:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3007.226952] Lustre: 101476:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3007.230647] Lustre: 101476:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 6a3d2e34-1794-4332-bf72-1b14a06f5f43@192.168.201.19@tcp [ 3007.239397] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3007.243931] Lustre: 101476:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@00000000dd252f93 x1875733246971264/t0(8589934769) o101->6a3d2e34-1794-4332-bf72-1b14a06f5f43@192.168.201.19@tcp:107/0 lens 592/0 e 0 to 0 dl 1788839727 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3007.274898] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3007.290819] Lustre: lustre-MDT0000: Denying connection for new client 6a3d2e34-1794-4332-bf72-1b14a06f5f43 (at 192.168.201.19@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3007.314530] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3007.343034] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3007.346904] Lustre: Skipped 2 previous similar messages [ 3007.379473] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3007.411438] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:77 to 0x280000401:577) [ 3007.416197] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:45 to 0x2c0000401:545) [ 3030.593862] Lustre: server umount lustre-OST0000 complete [ 3033.580560] LustreError: 97047:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3033.614295] LustreError: 97047:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 37 previous similar messages [ 3038.190812] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3042.533639] Lustre: server umount lustre-OST0001 complete [ 3048.425083] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3048.452839] Lustre: Skipped 7 previous similar messages [ 3048.458096] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3048.463734] Lustre: Skipped 4 previous similar messages [ 3053.991457] Lustre: server umount lustre-MDT0000 complete [ 3059.167651] LustreError: 97069:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788839768 with bad export cookie 10267178488232937720 [ 3059.188116] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3059.201403] LustreError: 97069:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3059.826153] Lustre: server umount lustre-MDT0001 complete [ 3070.973571] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 23:56:18 (1788839778) [ 3080.842081] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3080.859765] Lustre: *** cfs_fail_loc=197, val=0*** [ 3096.548917] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3101.728843] LustreError: 103449:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.119@tcp: failed processing log, type 4: rc = -110 [ 3106.915462] Lustre: *** cfs_fail_loc=197, val=0*** [ 3106.927239] Lustre: Skipped 65 previous similar messages [ 3127.585201] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3127.591088] Lustre: Skipped 4 previous similar messages [ 3136.196988] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 3145.818550] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3148.745213] Lustre: server umount lustre-OST0000 complete [ 3159.920845] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 23:57:46 (1788839866) [ 3185.851385] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3186.584759] LustreError: 105073:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3186.616426] LustreError: 105073:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 3193.770820] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 3205.272342] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3210.432497] LustreError: 105105:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3210.461842] LustreError: 105105:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3210.477991] LustreError: 105105:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 24, retries 0, failed: rc = -5 [ 3211.267314] Lustre: server umount lustre-MDT0000 complete [ 3255.267223] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_hostid [ 3266.372430] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 3312.363734] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3312.547977] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3312.573555] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3312.657491] Lustre: lustre-MDT0000: new disk, initializing [ 3312.773443] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3318.601364] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 3330.002861] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3330.176485] Lustre: 109287:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3330.492606] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3335.436992] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 3347.834202] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3355.661270] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3363.838732] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3364.198824] Lustre: 111267:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3369.711433] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3371.969524] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 3384.276820] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3391.206854] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3391.453950] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3399.795611] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3400.469892] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3405.793988] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3405.804723] LustreError: Skipped 2 previous similar messages [ 3405.805806] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3405.813118] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3405.813129] Lustre: Skipped 1 previous similar message [ 3405.856765] Lustre: Skipped 2 previous similar messages [ 3410.802523] Lustre: server umount lustre-OST0000 complete [ 3421.892102] Lustre: server umount lustre-MDT0000 complete [ 3426.807205] LustreError: 109279:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788840135 with bad export cookie 10267178488232948297 [ 3426.824382] LustreError: 109279:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3426.828384] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3427.365682] Lustre: server umount lustre-MDT0001 complete [ 3436.973565] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 00:02:24 (1788840144) [ 3447.134655] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 3450.144292] Key type lgssc unregistered [ 3450.380810] LNet: 113756:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3450.388703] LNetError: 113756:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3450.413455] LNet: Removed LNI 192.168.201.119@tcp [ 3451.365527] Key type .llcrypt unregistered [ 3451.369558] Key type ._llcrypt unregistered [ 3477.606123] Key type ._llcrypt registered [ 3477.609281] Key type .llcrypt registered [ 3477.740079] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_hostid [ 3496.190507] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 3497.741375] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3498.009737] alg: No test for adler32 (adler32-zlib) [ 3499.129550] Lustre: Lustre: Build Version: 2.17.57_113_gf5ca795 [ 3499.495816] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 3501.264202] Key type lgssc registered [ 3502.808146] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3550.944477] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3550.976897] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3552.311848] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3552.338836] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3552.424917] Lustre: lustre-MDT0000: new disk, initializing [ 3552.520686] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3552.536463] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3557.875355] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 3569.529097] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3569.765339] Lustre: lustre-OST0000: new disk, initializing [ 3569.768461] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3569.775281] Lustre: Skipped 1 previous similar message [ 3569.780925] Lustre: 117931:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3569.860714] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3577.941754] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 3578.022655] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3578.043433] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3578.114154] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3591.990136] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3592.217800] Lustre: lustre-OST0001: new disk, initializing [ 3592.223652] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3592.232242] Lustre: 118795:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3592.385666] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3599.005269] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3599.021861] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3599.082475] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3599.934272] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 3609.572092] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3609.590791] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3609.600397] Lustre: Skipped 1 previous similar message [ 3614.644082] LustreError: 119402:0:(ldlm_resource.c:1207:ldlm_resource_complain()) MGS: namespace resource [0x65727473756c:0x0:0x0].0x0 (0000000046189062) refcount nonzero (2) after lock cleanup; forcing cleanup. [ 3615.048524] Lustre: server umount lustre-MDT0000 complete [ 3623.205621] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3628.942181] LustreError: 117116:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788840338 with bad export cookie 6246377333763472674 [ 3628.960845] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3628.966722] LustreError: 117116:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3629.054621] Lustre: server umount lustre-OST0000 complete [ 3633.391956] Lustre: server umount lustre-OST0001 complete [ 3644.523462] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 00:05:51 (1788840351) [ 3674.794434] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 00:06:22 (1788840382) [ 3684.797024] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 3711.341259] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 00:06:58 (1788840418) [ 3756.645362] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_hostid [ 3768.174790] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 3817.039785] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3817.273320] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3817.299921] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3817.375463] Lustre: lustre-MDT0000: new disk, initializing [ 3817.464224] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3817.485545] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3823.985923] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 3838.164284] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3838.407552] Lustre: 126549:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3838.520815] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3838.529178] Lustre: Skipped 1 previous similar message [ 3838.693195] Lustre: lustre-MDT0001: new disk, initializing [ 3838.876828] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3838.988134] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3839.020918] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3845.815517] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 3858.414430] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3865.638588] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3874.469410] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3874.826420] Lustre: lustre-OST0000: new disk, initializing [ 3874.831423] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3874.846176] Lustre: 128522:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3874.960575] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3883.070318] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 3883.615135] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3883.627229] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3883.740186] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3894.954700] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3901.880909] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3902.253788] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3909.010690] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3909.399414] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3914.722761] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3914.725513] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3914.744198] LustreError: Skipped 1 previous similar message [ 3914.747303] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3914.782931] Lustre: Skipped 2 previous similar messages [ 3918.597792] Lustre: server umount lustre-OST0000 complete [ 3924.968707] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3924.990896] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3924.995215] Lustre: Skipped 2 previous similar messages [ 3930.094827] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3930.098317] Lustre: Skipped 1 previous similar message [ 3930.411168] Lustre: server umount lustre-MDT0000 complete [ 3934.181549] LustreError: 126562:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3934.196611] LustreError: 126562:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 3934.315745] LustreError: 126544:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788840643 with bad export cookie 6246377333763473892 [ 3934.326987] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3934.327980] LustreError: 126544:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3934.674651] Lustre: server umount lustre-MDT0001 complete [ 3943.791797] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 00:10:51 (1788840651) [ 3950.293370] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3950.804825] LustreError: 130935:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3950.824269] LustreError: 130935:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3950.958071] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3956.040122] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 3956.210465] LustreError: 130936:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3960.300259] LustreError: 130936:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3965.421342] LustreError: 130935:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3967.357413] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3972.769651] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 3981.013716] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3987.569949] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3994.754167] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3995.202719] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3995.212995] Lustre: Skipped 1 previous similar message [ 4002.390321] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4011.657033] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4020.086841] Lustre: *** cfs_fail_loc=159, val=0*** [ 4036.699415] Lustre: lustre-MDT0000: Client 0c79ce95-5f4e-4c0a-840e-54d50f94d90f (at 192.168.201.19@tcp) reconnecting [ 4038.946993] Lustre: *** cfs_fail_loc=159, val=0*** [ 4038.950158] Lustre: Skipped 6 previous similar messages [ 4054.578818] Lustre: lustre-MDT0000: Client 0c79ce95-5f4e-4c0a-840e-54d50f94d90f (at 192.168.201.19@tcp) reconnecting [ 4061.796957] Lustre: server umount lustre-OST0000 complete [ 4068.834393] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4068.837854] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4068.869614] Lustre: Skipped 2 previous similar messages [ 4079.072496] Lustre: 115134:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788840772/real 1788840772] req@0000000028f4869a x1875734910060928/t0(0) o13->lustre-OST0000-osc-MDT0001@0@lo:7/4 lens 224/368 e 0 to 1 dl 1788840788 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 4079.074240] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4079.104783] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4079.111026] Lustre: Skipped 4 previous similar messages [ 4080.096444] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4080.543183] Lustre: server umount lustre-MDT0000 complete [ 4084.197913] LustreError: 130930:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4084.225708] LustreError: 130930:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4084.324032] Lustre: 115134:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788840777/real 1788840777] req@00000000a44912a5 x1875734910063360/t0(0) o400->lustre-OST0000-osc-MDT0001@0@lo:28/4 lens 224/224 e 0 to 1 dl 1788840793 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 4084.700491] LustreError: 133534:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788840793 with bad export cookie 6246377333763474781 [ 4084.710343] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4084.716301] LustreError: 133534:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4085.050519] Lustre: server umount lustre-MDT0001 complete [ 4096.798376] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 4100.522383] Key type lgssc unregistered [ 4100.780853] LNet: 134710:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4100.790932] LNetError: 134710:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4100.817333] LNet: Removed LNI 192.168.201.119@tcp [ 4101.691411] Key type .llcrypt unregistered [ 4101.694871] Key type ._llcrypt unregistered [ 4128.403324] Key type ._llcrypt registered [ 4128.405950] Key type .llcrypt registered [ 4128.504166] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 4129.799498] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4129.811116] alg: No test for adler32 (adler32-zlib) [ 4130.823391] Lustre: Lustre: Build Version: 2.17.57_113_gf5ca795 [ 4131.064123] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 4132.712165] Key type lgssc registered [ 4134.218713] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4144.307935] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4144.345862] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4146.036060] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4150.994808] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4156.390524] LustreError: 136126:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4156.419469] LustreError: 136126:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4157.416609] LustreError: 136125:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4162.127610] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4162.624480] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4168.237458] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4177.659049] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4185.515834] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4193.369598] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4193.884525] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4201.314320] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4209.467870] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4217.470324] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4230.674899] Lustre: *** cfs_fail_loc=159, val=0*** [ 4246.027649] Lustre: lustre-MDT0000: Client 58ea273c-a8be-40c5-a4eb-de3d53ae9b10 (at 192.168.201.19@tcp) reconnecting [ 4249.390550] Lustre: *** cfs_fail_loc=159, val=0*** [ 4266.114260] Lustre: lustre-MDT0001: Client 58ea273c-a8be-40c5-a4eb-de3d53ae9b10 (at 192.168.201.19@tcp) reconnecting [ 4268.698742] Lustre: *** cfs_fail_loc=159, val=0*** [ 4268.711541] Lustre: Skipped 4 previous similar messages [ 4285.047336] Lustre: lustre-MDT0001: Client 58ea273c-a8be-40c5-a4eb-de3d53ae9b10 (at 192.168.201.19@tcp) reconnecting [ 4301.109373] Lustre: 139226:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4304.500843] Lustre: *** cfs_fail_loc=159, val=0*** [ 4304.507506] Lustre: Skipped 6 previous similar messages [ 4320.865659] Lustre: lustre-MDT0000: Client 532af308-ba6d-47f8-8066-907d3e62d8fc (at 192.168.201.19@tcp) reconnecting [ 4322.595079] Lustre: *** cfs_fail_loc=159, val=0*** [ 4322.602676] Lustre: Skipped 11 previous similar messages [ 4338.328045] Lustre: lustre-MDT0000: Client 532af308-ba6d-47f8-8066-907d3e62d8fc (at 192.168.201.19@tcp) reconnecting [ 4340.308596] Lustre: 139521:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4349.946107] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 4349.946366] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4349.980517] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4349.997127] LustreError: Skipped 2 previous similar messages [ 4350.964154] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4350.970872] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4350.971070] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4351.000460] Lustre: Skipped 1 previous similar message [ 4352.892887] Lustre: server umount lustre-OST0000 complete [ 4361.200767] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4361.206634] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4361.215601] Lustre: Skipped 1 previous similar message [ 4361.229789] Lustre: Skipped 1 previous similar message [ 4365.473425] Lustre: server umount lustre-MDT0000 complete [ 4366.325672] LustreError: 136125:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4366.351573] LustreError: 136125:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4370.722416] LustreError: 136106:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788841079 with bad export cookie 5829549088077589991 [ 4370.740860] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4370.751415] LustreError: 136106:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4371.515967] Lustre: server umount lustre-MDT0001 complete [ 4383.965495] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 4388.043736] Key type lgssc unregistered [ 4388.413769] LNet: 140579:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4388.423078] LNetError: 140579:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4388.444365] LNet: Removed LNI 192.168.201.119@tcp [ 4389.653732] Key type .llcrypt unregistered [ 4389.660622] Key type ._llcrypt unregistered [ 4418.557847] Key type ._llcrypt registered [ 4418.559715] Key type .llcrypt registered [ 4418.688822] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 4420.492769] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4420.624681] alg: No test for adler32 (adler32-zlib) [ 4421.856194] Lustre: Lustre: Build Version: 2.17.57_113_gf5ca795 [ 4422.092754] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 4423.739587] Key type lgssc registered [ 4425.934793] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4435.431139] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4435.447078] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4436.901090] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4441.042328] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4446.185723] LustreError: 141993:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4446.209953] LustreError: 141993:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4450.045950] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4450.736083] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4455.969608] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4465.157704] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4471.744514] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4480.299881] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4480.819769] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4481.864446] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4488.756935] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4495.177303] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4496.921489] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4504.651989] Lustre: 144575:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4508.048230] Lustre: 144624:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4509.679852] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4510.641399] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4514.471923] Lustre: 144771:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4515.892347] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4517.104869] Lustre: 144869:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4522.979840] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4522.990822] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4523.024937] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4526.050226] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4526.057511] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4526.071097] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4528.265208] Lustre: server umount lustre-OST0000 complete [ 4532.708886] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4532.725254] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4532.763197] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4532.769359] Lustre: Skipped 1 previous similar message [ 4536.291256] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4536.291761] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4539.126309] Lustre: server umount lustre-MDT0000 complete [ 4541.423875] LustreError: 141992:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4541.446305] LustreError: 141992:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4543.711879] LustreError: 141972:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788841252 with bad export cookie 11267785146832053318 [ 4543.726137] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4544.093150] Lustre: server umount lustre-MDT0001 complete [ 4560.349777] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 4564.620300] Key type lgssc unregistered [ 4564.967350] LNet: 145930:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4564.985982] LNetError: 145930:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4565.016421] LNet: Removed LNI 192.168.201.119@tcp [ 4566.054723] Key type .llcrypt unregistered [ 4566.063273] Key type ._llcrypt unregistered [ 4594.542647] Key type ._llcrypt registered [ 4594.548921] Key type .llcrypt registered [ 4594.695877] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 4596.692267] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4596.745787] alg: No test for adler32 (adler32-zlib) [ 4598.101938] Lustre: Lustre: Build Version: 2.17.57_113_gf5ca795 [ 4598.518794] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 4600.272837] Key type lgssc registered [ 4601.855434] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4610.978880] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4610.990177] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4612.442758] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4617.424194] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4622.824110] LustreError: 147341:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4622.860327] LustreError: 147341:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4627.944142] LustreError: 147342:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4628.140981] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4628.649664] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4634.796225] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4644.121202] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4651.341724] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4660.213205] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4660.662708] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4666.704612] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4672.962771] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4673.753905] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4682.193049] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4691.862967] Lustre: *** cfs_fail_loc=159, val=0*** [ 4707.396246] Lustre: lustre-MDT0000: Client fc9ac850-43b8-4804-bfb4-1cd0f4eae054 (at 192.168.201.19@tcp) reconnecting [ 4728.293544] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4728.299486] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4728.327265] Lustre: Skipped 1 previous similar message [ 4728.329335] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4728.362267] Lustre: Skipped 1 previous similar message [ 4731.694726] Lustre: server umount lustre-OST0000 complete [ 4738.531586] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4738.548683] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4738.560275] Lustre: Skipped 1 previous similar message [ 4738.590564] Lustre: Skipped 1 previous similar message [ 4743.653977] LustreError: 148292:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4743.671210] LustreError: 148292:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4743.997551] Lustre: server umount lustre-MDT0000 complete [ 4748.770559] LustreError: 147342:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4748.787989] LustreError: 147342:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4749.292369] LustreError: 147321:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788841458 with bad export cookie 9863709953447404524 [ 4749.293252] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4749.308339] LustreError: 147321:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4749.815142] Lustre: server umount lustre-MDT0001 complete [ 4762.294869] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 4765.538937] Key type lgssc unregistered [ 4765.994993] LNet: 151042:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4766.008533] LNetError: 151042:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4766.048362] LNet: Removed LNI 192.168.201.119@tcp [ 4767.147371] Key type .llcrypt unregistered [ 4767.152640] Key type ._llcrypt unregistered [ 4802.228567] Key type ._llcrypt registered [ 4802.229839] Key type .llcrypt registered [ 4802.339917] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 4802.890521] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4802.941758] alg: No test for adler32 (adler32-zlib) [ 4804.440532] Lustre: Lustre: Build Version: 2.17.57_113_gf5ca795 [ 4804.810389] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 4806.528225] Key type lgssc registered [ 4808.387127] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4818.306754] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4818.322867] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4819.913973] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4825.173472] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4829.163327] LustreError: 152451:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4829.181624] LustreError: 152451:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4834.274473] LustreError: 152452:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4834.627734] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4835.180623] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4840.478561] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4848.249154] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4853.495125] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4862.471610] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4862.923223] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4864.006101] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4871.681694] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 4880.384854] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4881.407426] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4902.643130] Lustre: lustre-OST0000: evicting ef7d2e88-b84e-4d72-9be8-d93aebbf2f7d (at 192.168.201.19@tcp) by administrative request [ 4902.656619] Lustre: Skipped 1 previous similar message [ 4935.651203] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4935.667136] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4935.691454] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4936.674485] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4936.677279] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4936.687900] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4936.719751] Lustre: Skipped 1 previous similar message [ 4938.698163] Lustre: server umount lustre-OST0000 complete [ 4946.914663] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4946.924121] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4946.935043] Lustre: Skipped 1 previous similar message [ 4946.953816] Lustre: Skipped 1 previous similar message [ 4949.853017] Lustre: server umount lustre-MDT0000 complete [ 4952.038971] LustreError: 152448:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4952.066377] LustreError: 152448:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4954.573652] LustreError: 152433:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788841663 with bad export cookie 11335077220119071445 [ 4954.601065] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4955.132979] Lustre: server umount lustre-MDT0001 complete [ 4967.642598] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 4970.558877] Key type lgssc unregistered [ 4970.833493] LNet: 156517:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4970.842330] LNetError: 156517:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4970.862209] LNet: Removed LNI 192.168.201.119@tcp [ 4971.701728] Key type .llcrypt unregistered [ 4971.704678] Key type ._llcrypt unregistered [ 5063.923581] Key type ._llcrypt registered [ 5063.927291] Key type .llcrypt registered [ 5064.060674] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 5065.679182] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5065.733340] alg: No test for adler32 (adler32-zlib) [ 5066.888385] Lustre: Lustre: Build Version: 2.17.57_113_gf5ca795 [ 5067.178248] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 5068.848459] Key type lgssc registered [ 5070.219403] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5078.220823] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5078.237394] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5079.600267] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5084.268381] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 5089.765608] LustreError: 159007:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5089.788774] LustreError: 159007:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5094.594195] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5095.023182] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5101.000962] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 5109.804757] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5117.489570] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5125.384747] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5125.801625] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5132.940702] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 5140.161471] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5140.652484] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5149.372892] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 5160.929583] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5160.935313] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5160.944238] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5165.029292] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5165.029808] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5165.043813] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5166.189932] Lustre: server umount lustre-OST0000 complete [ 5171.681989] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5171.697066] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5171.714674] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5171.717379] Lustre: Skipped 1 previous similar message [ 5175.265435] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5175.267307] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5175.308900] Lustre: Skipped 1 previous similar message [ 5176.964276] Lustre: server umount lustre-MDT0000 complete [ 5180.395102] LustreError: 159001:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5180.425912] LustreError: 159001:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 5181.539239] LustreError: 158987:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788841890 with bad export cookie 1092690968212674710 [ 5181.572872] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5182.206233] Lustre: server umount lustre-MDT0001 complete [ 5194.294769] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 5197.972574] Key type lgssc unregistered [ 5198.413129] LNet: 162569:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5198.420541] LNetError: 162569:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5198.448201] LNet: Removed LNI 192.168.201.119@tcp [ 5199.612187] Key type .llcrypt unregistered [ 5199.615935] Key type ._llcrypt unregistered [ 5229.647994] Key type ._llcrypt registered [ 5229.651722] Key type .llcrypt registered [ 5229.752901] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 5231.053571] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5231.170878] alg: No test for adler32 (adler32-zlib) [ 5232.230798] Lustre: Lustre: Build Version: 2.17.57_113_gf5ca795 [ 5232.526987] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 5234.210579] Key type lgssc registered [ 5235.876302] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5255.443809] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 00:32:42 (1788841962) [ 5274.507085] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_hostid [ 5286.979778] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 5337.427924] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5337.485716] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5339.105238] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5339.147066] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5339.261353] Lustre: lustre-MDT0000: new disk, initializing [ 5339.367629] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5339.406746] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5345.890881] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 5360.127745] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5360.276658] Lustre: 166831:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5360.386563] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5360.391474] Lustre: Skipped 1 previous similar message [ 5360.497411] Lustre: lustre-MDT0001: new disk, initializing [ 5360.651583] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5360.713044] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5360.752699] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5366.689991] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 5377.153601] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5383.801340] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5391.959088] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5392.211105] Lustre: lustre-OST0000: new disk, initializing [ 5392.215545] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5392.222984] Lustre: 168804:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5392.338960] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5400.265951] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5400.274692] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 5400.277684] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5400.508746] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5410.463618] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5426.146977] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5426.154278] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5426.164748] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5426.178761] Lustre: Skipped 1 previous similar message [ 5426.217285] Lustre: Skipped 1 previous similar message [ 5428.261215] Lustre: server umount lustre-OST0000 complete [ 5436.389635] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5436.393619] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5436.404736] Lustre: Skipped 1 previous similar message [ 5439.429861] Lustre: server umount lustre-MDT0000 complete [ 5443.219778] LustreError: 166823:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788842152 with bad export cookie 15499769217056711682 [ 5443.223990] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5443.235452] LustreError: 166823:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5443.488395] Lustre: server umount lustre-MDT0001 complete [ 5456.759803] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 5459.610287] Key type lgssc unregistered [ 5460.108492] LNet: 170708:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5460.130164] LNetError: 170708:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5460.157302] LNet: Removed LNI 192.168.201.119@tcp [ 5461.234607] Key type .llcrypt unregistered [ 5461.236264] Key type ._llcrypt unregistered [ 5491.389909] Key type ._llcrypt registered [ 5491.392920] Key type .llcrypt registered [ 5491.510963] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 5493.175191] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5493.224819] alg: No test for adler32 (adler32-zlib) [ 5494.515722] Lustre: Lustre: Build Version: 2.17.57_113_gf5ca795 [ 5494.893289] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 5496.600238] Key type lgssc registered [ 5498.217286] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5507.015982] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5507.045344] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5508.685569] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5513.235796] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 5518.825264] LustreError: 172120:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5518.844280] LustreError: 172120:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5522.533741] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5522.892767] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5527.172608] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 5534.915778] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5540.612900] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5547.718493] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5548.162158] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5553.282276] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5555.217960] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 5562.981893] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5573.613479] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5573.634517] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5573.658688] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5573.672383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.680348] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5574.601573] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5574.621533] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5574.631422] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5574.639188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.649836] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5575.706533] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5575.718360] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5575.727219] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5575.733036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.746463] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5576.837167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.746493] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5577.750212] Lustre: Skipped 1 previous similar message [ 5577.759934] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5577.773450] Lustre: Skipped 1 previous similar message [ 5577.776708] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5577.780502] Lustre: Skipped 1 previous similar message [ 5577.784667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.790360] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5577.792944] Lustre: Skipped 1 previous similar message [ 5578.694405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.733526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.839173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.949570] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5581.958407] Lustre: Skipped 3 previous similar messages [ 5581.986406] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5581.994575] Lustre: Skipped 3 previous similar messages [ 5582.007755] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5582.010948] Lustre: Skipped 3 previous similar messages [ 5582.017883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.027903] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5582.041963] Lustre: Skipped 3 previous similar messages [ 5583.082029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.457884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.395840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.526291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.489530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.617738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.524242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.743109] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5590.745957] Lustre: Skipped 7 previous similar messages [ 5590.773456] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5590.788479] Lustre: Skipped 7 previous similar messages [ 5590.793270] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5590.796277] Lustre: Skipped 7 previous similar messages [ 5590.804666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.814372] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5590.819734] Lustre: Skipped 7 previous similar messages [ 5592.025459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.091751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.035686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.323984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.277487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.323854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.277698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.406407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.407335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.553965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.384849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.697736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.960617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.063117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.257670] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5607.261762] Lustre: Skipped 14 previous similar messages [ 5607.278160] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5607.284813] Lustre: Skipped 14 previous similar messages [ 5607.289847] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5607.294285] Lustre: Skipped 14 previous similar messages [ 5607.297553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.304742] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5607.310330] Lustre: Skipped 14 previous similar messages [ 5608.495681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.762731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.116452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.462360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.074404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.312036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.597321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.627945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.118669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.447445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.867986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.316590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.712573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.150472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.404869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.615450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.800169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.077217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.264335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.325744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.534593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.553159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.907582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.006344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.423617] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5639.430419] Lustre: Skipped 24 previous similar messages [ 5639.449166] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5639.462605] Lustre: Skipped 24 previous similar messages [ 5639.469325] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5639.476740] Lustre: Skipped 24 previous similar messages [ 5639.492979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.506721] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5639.533433] Lustre: Skipped 24 previous similar messages [ 5640.774038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.946162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.967567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.280441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.184298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.402464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.674313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.023594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.319096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.666826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.673135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.842358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.081279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.316293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.447289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.548299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.707976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.795307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.986719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.153534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.232772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.463430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.701556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.820171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.713494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.732442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.630651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.857831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.778790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.088116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.079605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.194240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.240088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.479325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.540962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.661868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.847806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.875621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.942440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.931256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.138730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.143937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.196539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.205526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.193328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.297524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.272404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.385320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.484579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.564524] LustreError: 172140:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5695.583049] LustreError: 172140:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5695.598023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.694602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.854326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.924838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.076041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.218102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.335200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.466642] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5703.481101] Lustre: Skipped 56 previous similar messages [ 5703.506068] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5703.514963] Lustre: Skipped 56 previous similar messages [ 5703.519976] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5703.524724] Lustre: Skipped 56 previous similar messages [ 5703.531950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.544931] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5703.547752] Lustre: Skipped 56 previous similar messages [ 5704.711357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.947275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.147813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.129431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.129578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.172346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.303499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.500824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.644652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.762214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.718709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.613930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.569688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.799294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.087410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.131864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.241936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.282627] LustreError: 172140:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5723.290450] LustreError: 172140:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5723.295503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.386884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.533840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.459192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.360111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.372744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.373428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.324859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.352459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.753156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.193828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.472835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.455490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.590201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.568872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.462033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.328476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.463552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.444061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.430585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.396094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.419389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.558337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.475725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.631980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.567639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.521896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.703838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.976967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.996446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.986510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.986991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.063245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.063687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.108834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.042619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.001821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.084084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.184218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.311875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.404165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.572899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.913280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.986507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.187080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.297646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.532800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.298689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.345279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.725597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.887783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.932091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.427673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.620114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.787039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.822807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.800250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.933897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.263785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.477754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.532527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.625449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.643931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.801143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.967918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.998964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.071696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.043071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.175063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.392564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.372830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.770923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.900081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.887732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.038721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.936603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.098605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.036127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.185669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.663398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.838625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.939238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.273584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.487967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.527458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.642307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.747181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.751889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.894332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.870660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.825567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.749203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.850864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.820162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.018070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.063410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.928211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.837247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.982405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.149186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.302831] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5832.307479] Lustre: Skipped 117 previous similar messages [ 5832.326616] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5832.332597] Lustre: Skipped 117 previous similar messages [ 5832.335840] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5832.338595] Lustre: Skipped 117 previous similar messages [ 5832.340862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.346023] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5832.353284] Lustre: Skipped 117 previous similar messages [ 5833.760324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.853065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.118581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.454438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.438632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.945241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.060392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.019351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.532327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.773757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.722940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.985917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.345871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.659102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.028340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.292216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.430977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.739361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.061672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.221399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.308537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.541919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.829292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.017615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.212150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.203648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.258643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.489273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.495194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.533950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.424519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.510541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.447861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.796847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.884350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.791531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.868763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.103753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.366623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.504899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.568928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.640076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.982808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.148735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.494614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.568113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.502686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.760148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.644348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.728642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.784509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.743029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.001639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.011784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.868936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.933499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.184164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.259986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.271811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.356114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.510034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.521924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.503495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.505908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.338675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.407100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.542197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.703596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.635794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.711540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.639348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.652235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.643788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.824823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.644427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.710836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.474686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.410226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.331691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.398699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.376213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.439882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.253894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.123908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.183108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.235217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.249510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.226702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.040129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.740601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.808615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.805540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.877141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.039779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.113565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.330397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.521613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.912407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.069740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.235412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.470132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.785662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.571538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.348215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.032134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.148816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.894892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.064157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.323585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.612596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.788180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.869589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.906963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.777481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.013096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.357084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.362885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.115200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.566688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.649270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.447586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.246761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.164990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.318336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.337479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.260919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.172384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.300963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.383025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.433749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.602292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.730835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.885867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.910032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.072410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.130705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.134909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.253302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.188990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.985055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.827605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.957658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.896069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.096132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.223492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.422329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.459216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.678757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.851135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.892160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.997914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.845545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.979412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.845859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.088952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.369825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.362408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.346931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.397201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.426741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.355079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.514492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.522651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.482140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.701432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.890936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.876660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.961788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.895488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.705498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.606808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.602968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.645384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.685106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.794364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.870526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.878232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.997454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.975528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.171474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.307508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.268321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.441839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.408080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.352201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.443189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.508304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.421797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.334359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.195076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.196477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.264950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.229574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.364157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.295960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.266698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.219988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.257720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.291194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.150018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.027244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.213422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.160390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.081524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.560449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.537643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.855847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.052472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.271487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.646614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.897745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.082945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.228743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.518067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.569230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.694888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.222557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.759337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.589878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.327411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.090929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.658034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.155264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.670764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.009925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.645824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.005511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.014817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.404399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.677295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.724778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.723265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.731418] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6088.734285] Lustre: Skipped 232 previous similar messages [ 6088.752418] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6088.765278] Lustre: Skipped 232 previous similar messages [ 6088.770857] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6088.775523] Lustre: Skipped 232 previous similar messages [ 6088.779780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.802923] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6088.805436] Lustre: Skipped 232 previous similar messages [ 6089.771405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.775672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.069946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.029335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.200618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.155928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.280754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.359967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.489451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.515471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.730383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.961183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.791830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.799946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.800911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.063218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.295269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.383696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.530456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.845414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.744420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.848107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.844059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.039650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.995369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.222510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.191195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.413579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.423814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.503574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.437787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.455347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.656594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.673573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.683060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.967823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.191287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.273861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.478375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.591429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.673986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.616627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.630539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.802854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.995052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.329429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.310577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.508640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.619702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.720610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.944876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.142446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.322389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.694272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.706495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.841185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.065368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.082412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.164676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.386197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.470900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.462348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.803339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.706273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.961212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.255741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.521807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.628065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.772868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.938767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.177632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.226516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.431586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.415554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.336832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.251947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.295559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.243611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.215901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.084655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.989340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.854723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.990663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.020030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.965042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.989343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.873852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.689388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.592577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.668464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.657274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.617977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.715191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.815661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.924701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.015483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.271516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.280931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.319182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.210797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.456754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.478509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.564628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.564543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.663703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.749630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.053076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.239674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.197981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.526098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.540453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.589865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.730132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.939589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.115867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.474339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.444685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.620929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.531801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.509373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.450483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.439314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.536251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.632332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.711660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.494416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.444407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.348444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.253304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.324533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.460795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.623862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.734883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.754573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.732852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.658207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.795670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.930161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.954338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.036238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.111457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.173349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.302927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.370387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.326324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.490102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.553192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.710602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.858603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.121254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.353885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.492143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.687456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.849859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.038966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.218253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.345505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.517040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.666301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.919666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.116793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.303313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.360898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.415361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.614471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.652272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.847282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.107381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.146733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.297222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.418520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.437387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.504966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.130437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.191965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.121450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.175429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.286875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.212404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.277825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.456819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.519111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.785603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.218088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.390267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.406573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.655812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.508090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.409479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.374882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.394298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.615476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.590383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.855947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.668081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.437537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.312696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.163632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.379209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.778136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.955082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.402385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.426471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.630638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.467639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.338799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.421571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.677745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.984767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.279434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.999806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.100801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.289074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.412060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.341561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.371524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.604433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.711522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.183329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.481450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.617809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6331.924918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.114374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.041418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.078432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.051482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.449817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.114873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.777070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6342.314352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6343.555968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.890487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.287874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.756682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6348.952952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.572861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.238292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.511479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6354.857194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.781088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.983618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.118525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.552830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.778983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6361.967786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.285990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.612975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.429663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.484439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.169692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.264651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.462270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.605647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.906778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6373.104928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.318878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.362225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.548386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6377.719177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.792914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.957552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.251698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6382.408066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.509724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6384.768419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6385.892368] LustreError: 172140:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6385.905665] LustreError: 172140:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6385.937222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.018711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6388.142701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6389.220213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6390.473168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.656073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6392.665327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.845275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6394.924153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6395.808048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6396.856659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.817385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.771230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6399.594888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.771508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6401.834538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.773523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6404.175535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.171332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6406.166790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6407.237542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6408.200286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6409.246415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.578583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.509684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6412.711441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.521326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6414.551565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.539941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.491601] LustreError: 172140:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6416.497927] LustreError: 172140:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6416.514933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.594728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6418.868613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.651976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.662559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6421.971657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6423.053424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.088151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.465704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6426.819064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6428.038454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6429.443993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.499758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.651982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6432.681510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6433.781211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6434.903608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.877287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6437.163963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.045620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6439.416910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6440.694187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.266858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6443.756325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6444.804301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.735323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6446.958554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6448.061092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6449.145053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6450.240426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.082202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.961831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6452.749052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6453.869140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.056364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6456.057174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6457.050722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6458.204893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6459.410247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6460.980861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6462.184274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6463.415722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6464.568655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6465.691077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6466.861507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6467.682758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6468.835392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6469.755175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6470.643578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6471.459712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6472.379599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6473.186289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6474.117762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6475.142130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6476.234278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6477.123369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6478.040449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6478.907347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6479.732581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6480.618204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6481.757732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6482.797710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6484.436422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6485.751343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6487.327108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6488.395462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6489.763634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6491.041520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6492.033074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6493.669596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6494.939402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6496.180842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6497.575242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6499.274658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6500.775419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6503.102834] LustreError: 172140:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6503.125563] LustreError: 172140:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6503.145819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6504.757673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6505.652848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6506.459759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6507.542132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6508.866074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6509.833794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6510.915794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6512.152470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6513.101449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6514.396166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6515.481443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6516.632497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6517.865309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6518.808033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6519.919219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6520.984657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6521.959204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6523.141630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6524.025598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6525.033188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6525.873554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6526.820862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6527.772493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6528.954812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6530.160851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6531.417587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6532.572118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6533.710293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6534.869122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6536.057426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6537.385536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6538.498937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6539.603148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6540.941829] LustreError: 172140:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6540.960305] LustreError: 172140:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6540.970844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6542.158960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6543.415444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6544.733151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6546.180650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6547.112516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6548.361589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6549.946358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6551.348696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6552.608764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6553.564696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6554.713869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6555.969226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6557.096376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6558.325598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6559.674474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6560.871720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6562.125494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6563.506449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6564.929480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6566.244103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6567.479922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6568.833698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6570.099638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6571.427977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6572.630129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6573.735068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6575.007919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6576.115827] LustreError: 172140:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6576.134724] LustreError: 172140:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6576.154506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6577.314656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6578.411431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6579.280909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6580.345053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6581.223173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6582.618525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6583.865956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6584.900455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6586.030347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6587.124564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6588.469174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6589.377944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6590.409307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6591.270724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6592.546191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6593.483095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6594.543209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6595.289362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6596.341109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6597.500888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6598.710849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6599.735061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6600.970465] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6600.979815] Lustre: Skipped 454 previous similar messages [ 6601.005565] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6601.013803] Lustre: Skipped 454 previous similar messages [ 6601.017508] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6601.019763] Lustre: Skipped 454 previous similar messages [ 6601.023061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6601.037565] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6601.048368] Lustre: Skipped 454 previous similar messages [ 6602.228841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6603.620798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6604.689097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6606.044245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6607.232121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6608.139544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6609.410558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6610.426924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6611.409302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6612.509559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6613.754354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6614.798904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6616.366846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6617.726757] LustreError: 172140:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6617.760458] LustreError: 172140:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6617.795312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6619.076703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6620.396580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6621.629504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6622.612824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6624.115734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6624.866157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6626.032213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6627.143401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6628.614432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6629.801309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6630.852093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6632.112119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6633.258906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6634.196908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6635.107910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6635.979328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6636.998934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6637.842348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6638.895064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6639.874597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6640.878756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6641.949435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6643.017902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6644.030329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6645.054807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6646.206609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6647.461101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6648.510543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6649.646450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6650.820931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6651.940114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6653.242546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6654.395246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6655.239687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6656.406321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6657.563297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6658.636291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6659.604716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6661.091338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6662.515561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6663.609803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6664.920726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6666.202098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6667.369190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6668.528758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6669.770246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6671.135197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6672.242478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6673.543338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6674.684836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6676.104340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6677.568342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6679.140101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6680.361421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6681.725391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6683.420185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6684.696764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6685.839582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6687.648560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6688.870558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6690.257934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6691.285653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6692.799734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6694.074662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6695.067627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6696.243931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7164.791903] LustreError: 173942:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 7164.807182] LustreError: 172140:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7846.375209] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7846.381575] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7846.404224] Lustre: Skipped 80 previous similar messages [ 7846.428306] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7847.399366] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7847.413496] Lustre: Skipped 1 previous similar message [ 7852.517236] LustreError: 173942:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7852.540476] LustreError: 173942:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7853.018148] Lustre: server umount lustre-OST0000 complete [ 7862.779710] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7862.783252] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7862.800300] Lustre: Skipped 1 previous similar message [ 7865.249849] Lustre: server umount lustre-MDT0000 complete [ 7867.872960] LustreError: 172120:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7867.897214] LustreError: 172120:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7869.592602] LustreError: 172103:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788844578 with bad export cookie 8965736032682122792 [ 7869.598420] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7869.599500] LustreError: 172103:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7870.140728] Lustre: server umount lustre-MDT0001 complete [ 7883.556657] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 7886.796860] Key type lgssc unregistered [ 7887.034859] LNet: 227420:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7887.041761] LNetError: 227420:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7887.067904] LNet: Removed LNI 192.168.201.119@tcp [ 7888.034108] Key type .llcrypt unregistered [ 7888.035961] Key type ._llcrypt unregistered [ 7916.871520] Key type ._llcrypt registered [ 7916.875756] Key type .llcrypt registered [ 7917.007404] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 7918.317806] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7918.598205] alg: No test for adler32 (adler32-zlib) [ 7919.635876] Lustre: Lustre: Build Version: 2.17.57_113_gf5ca795 [ 7919.956336] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 7921.688202] Key type lgssc registered [ 7922.991491] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7932.891787] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7932.921598] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7934.465636] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7940.210612] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 7944.677258] LustreError: 228833:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7944.709515] LustreError: 228833:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7949.800666] LustreError: 228834:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7950.504219] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7951.067237] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7956.616877] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 7966.391631] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7973.030679] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7981.478429] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7981.965237] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7989.124501] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 7997.625952] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8004.471172] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 8008.269443] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 8010.433948] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8010.445164] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 8010.452738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8010.475422] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 8010.475613] LustreError: 228853:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 8014.821959] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8014.826760] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8014.853055] Lustre: Skipped 1 previous similar message [ 8014.874489] Lustre: Skipped 2 previous similar messages [ 8018.861773] Lustre: server umount lustre-OST0000 complete [ 8030.537631] LustreError: 228853:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 8053.556734] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8054.055714] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8055.090738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8055.106744] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8055.119749] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 8055.148142] Lustre: Skipped 1 previous similar message [ 8060.898781] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 8069.197328] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8070.891583] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 8077.542474] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff94bd5141c000.ost_server_uuid 50 [ 8079.441426] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff94bd5141c000.ost_server_uuid in IDLE state after 0 sec [ 8084.969440] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8084.991409] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8085.006648] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8085.985156] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8091.103372] Lustre: server umount lustre-OST0000 complete [ 8096.231129] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8096.248067] Lustre: Skipped 1 previous similar message [ 8096.258198] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8096.268561] Lustre: Skipped 3 previous similar messages [ 8101.348849] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8101.356131] Lustre: Skipped 1 previous similar message [ 8102.051929] Lustre: server umount lustre-MDT0000 complete [ 8106.268843] LustreError: 232069:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788844815 with bad export cookie 13032702979617391393 [ 8106.270666] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8106.277498] LustreError: 232069:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8106.588540] Lustre: server umount lustre-MDT0001 complete [ 8119.168838] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 8122.553991] Key type lgssc unregistered [ 8122.929117] LNet: 234044:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8122.934882] LNetError: 234044:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8124.009559] LNet: Removed LNI 192.168.201.119@tcp [ 8125.072901] Key type .llcrypt unregistered [ 8125.077097] Key type ._llcrypt unregistered [ 8161.499300] Key type ._llcrypt registered [ 8161.504442] Key type .llcrypt registered [ 8161.672411] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 8163.082512] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8163.234718] alg: No test for adler32 (adler32-zlib) [ 8164.455890] Lustre: Lustre: Build Version: 2.17.57_113_gf5ca795 [ 8164.833024] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 8166.576202] Key type lgssc registered [ 8168.143655] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8177.786969] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8177.801417] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8179.339428] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8183.695310] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 8188.907958] LustreError: 235772:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8188.936710] LustreError: 235772:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8192.316022] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8192.719928] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8197.465769] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 8200.513473] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 8206.566052] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8206.847385] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8213.119298] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 8223.009876] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8223.259580] Lustre: lustre-OST0001: new disk, initializing [ 8223.267501] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8223.278266] Lustre: 238079:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8223.480961] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8224.520555] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 8229.072549] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8229.088767] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8229.244286] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8231.949643] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 8243.305789] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8249.938249] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 8265.198190] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8265.209059] LustreError: 235767:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8265.209702] Lustre: Skipped 2 previous similar messages [ 8265.229614] LustreError: 235767:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8265.279401] Lustre: server umount lustre-MDT0000 complete [ 8269.195235] LustreError: 238851:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788844978 with bad export cookie 11871543607089983338 [ 8269.224162] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8269.227401] LustreError: 238851:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8269.537725] Lustre: server umount lustre-MDT0001 complete [ 8284.644873] Lustre: server umount lustre-OST0000 complete [ 8286.433244] Lustre: 235035:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788844979/real 1788844979] req@ffff8e0403e75c00 x1875739801646848/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788844995 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8286.485385] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8286.494667] Lustre: Skipped 1 previous similar message [ 8288.993686] Lustre: server umount lustre-OST0001 complete [ 8315.386724] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 8328.050232] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8328.506072] LustreError: 241188:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8328.541183] LustreError: 241188:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8328.608471] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8333.604718] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 8333.800047] LustreError: 241189:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8338.928895] LustreError: 241189:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8343.561524] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8343.906474] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8348.817408] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 8351.999579] Lustre: 242332:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8359.519994] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8360.052820] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8367.436815] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 8368.242822] LustreError: 242684:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8368.262827] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:129) [ 8374.805832] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 8377.724520] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8383.014598] Lustre: mylustre-OST0001: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:33) [ 8385.720576] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 8394.517411] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8398.839093] Lustre: 244205:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8410.594144] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8410.616748] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8410.643950] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8413.669604] Lustre: mylustre-MDT0000-lwp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8413.675947] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8413.692956] Lustre: Skipped 1 previous similar message [ 8413.712874] Lustre: Skipped 1 previous similar message [ 8418.823925] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8418.832650] Lustre: Skipped 2 previous similar messages [ 8423.392712] Lustre: mylustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8423.755852] Lustre: server umount mylustre-MDT0000 complete [ 8423.918083] LustreError: 241924:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8423.961782] LustreError: 241924:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 8430.408518] LustreError: 241165:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788845139 with bad export cookie 11871543607089986201 [ 8430.411058] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8430.417815] LustreError: 241165:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 8431.491727] Lustre: server umount mylustre-MDT0001 complete [ 8446.271995] Lustre: server umount mylustre-OST0000 complete [ 8450.020469] Lustre: 235036:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788845143/real 1788845143] req@ffff8e053ce87100 x1875739801715968/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788845159 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8450.070704] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8450.098909] Lustre: Skipped 1 previous similar message [ 8451.387550] Lustre: server umount mylustre-OST0001 complete [ 8475.185127] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 8487.475853] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8488.209834] LustreError: 246515:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8488.226183] LustreError: 246515:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 8488.306362] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8488.319457] Lustre: Skipped 1 previous similar message [ 8494.031458] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 8502.352281] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8508.165076] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 8511.553108] Lustre: 247657:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8518.939324] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8526.651860] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 8527.487861] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 8534.042106] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 8536.755462] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8542.256396] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:65) [ 8545.763118] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 8554.843399] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8559.510158] Lustre: 249528:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8569.315615] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8569.331934] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8569.359772] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 8569.377639] Lustre: Skipped 1 previous similar message [ 8575.003958] Lustre: server umount tfs-MDT0000 complete [ 8578.020488] LustreError: 249779:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8578.041679] LustreError: 249779:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [ 8579.146063] LustreError: 246491:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788845288 with bad export cookie 11871543607089988378 [ 8579.146963] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8579.157990] LustreError: 246491:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8579.527638] Lustre: server umount tfs-MDT0001 complete [ 8593.925060] Lustre: server umount tfs-OST0000 complete [ 8608.710466] Lustre: server umount tfs-OST0001 complete [ 8634.377983] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 8645.650283] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8646.289724] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8646.294084] Lustre: Skipped 3 previous similar messages [ 8651.146385] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 8660.409752] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8666.807374] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 8670.976264] Lustre: 252925:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8679.725165] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8685.434574] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 8687.358846] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 8691.708249] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 8696.563451] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8701.966628] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:97) [ 8703.157608] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 8711.470799] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8716.135543] Lustre: 254799:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8725.548808] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 01:30:32 (1788845432) [ 8742.890083] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8742.892741] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8742.916255] Lustre: Skipped 6 previous similar messages [ 8742.916926] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8742.945110] Lustre: Skipped 7 previous similar messages [ 8748.754836] Lustre: server umount lustre-MDT0000 complete [ 8751.017808] LustreError: 251783:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8751.043379] LustreError: 251783:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 10 previous similar messages [ 8752.910823] LustreError: 254109:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788845462 with bad export cookie 11871543607089990296 [ 8752.918986] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8752.926080] LustreError: 254109:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8756.221644] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8756.230260] Lustre: Skipped 5 previous similar messages [ 8759.606200] Lustre: server umount lustre-MDT0001 complete [ 8770.521949] Lustre: server umount lustre-OST0000 complete [ 8781.146116] Lustre: server umount lustre-OST0001 complete [ 8789.359499] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_hostid [ 8800.431357] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 8839.832929] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8840.221598] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8840.319638] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8840.424444] Lustre: lustre-MDT0000: new disk, initializing [ 8840.553615] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 8840.558792] Lustre: Skipped 3 previous similar messages [ 8840.576923] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8845.217611] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 8859.400243] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8859.631740] Lustre: 257986:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8859.718768] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8859.725236] Lustre: Skipped 1 previous similar message [ 8859.903187] Lustre: lustre-MDT0001: new disk, initializing [ 8860.105840] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8860.118486] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8865.777523] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 8878.233636] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8878.569417] Lustre: lustre-OST0000: new disk, initializing [ 8878.574923] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8878.584655] Lustre: 259571:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8880.497378] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8880.529188] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8880.654320] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8886.630557] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 8897.978098] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8898.219171] Lustre: lustre-OST0001: new disk, initializing [ 8898.222292] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8898.226334] Lustre: 260443:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8900.087224] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8900.097691] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8900.283324] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8905.512542] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 8921.104023] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 01:33:48 (1788845628) [ 8942.159119] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 01:34:09 (1788845649) [ 8957.412618] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8957.415210] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8957.426206] LustreError: Skipped 1 previous similar message [ 8957.441375] Lustre: Skipped 2 previous similar messages [ 8957.443629] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8957.466302] Lustre: Skipped 3 previous similar messages [ 8963.551340] Lustre: server umount lustre-OST0000 complete [ 8975.341701] Lustre: server umount lustre-MDT0000 complete [ 8979.956206] LustreError: 260442:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788845689 with bad export cookie 11871543607089992074 [ 8979.963241] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8979.979186] LustreError: 260442:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8986.784939] Lustre: server umount lustre-MDT0001 complete [ 9001.601854] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 9078.785920] Lustre: server umount lustre-OST0001 complete [ 9088.814772] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_hostid [ 9108.938418] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 9148.613360] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9148.792829] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9148.831057] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9148.876660] Lustre: lustre-MDT0000: new disk, initializing [ 9148.916573] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9148.920487] Lustre: Skipped 3 previous similar messages [ 9148.929561] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9152.789327] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 9165.355572] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9165.531311] Lustre: 265328:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9165.609613] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9165.614278] Lustre: Skipped 1 previous similar message [ 9165.757692] Lustre: lustre-MDT0001: new disk, initializing [ 9166.014973] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9166.042469] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9172.551753] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 9183.969805] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9190.647422] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9198.760937] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9199.252985] Lustre: lustre-OST0000: new disk, initializing [ 9199.264433] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9199.274468] Lustre: 267296:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9200.590395] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9200.616305] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9200.846387] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9206.832843] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 9219.042551] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9226.217341] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9226.231992] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9226.253386] Lustre: Skipped 5 previous similar messages [ 9226.269849] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9226.284154] Lustre: Skipped 7 previous similar messages [ 9230.917401] Lustre: server umount lustre-OST0000 complete [ 9241.537472] Lustre: server umount lustre-MDT0000 complete [ 9242.084408] LustreError: 265338:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9242.110759] LustreError: 265338:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 14 previous similar messages [ 9246.078555] LustreError: 267306:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788845955 with bad export cookie 11871543607089993537 [ 9246.080315] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9246.088647] LustreError: 267306:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9246.422457] Lustre: server umount lustre-MDT0001 complete [ 9260.207833] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 9263.622973] Key type lgssc unregistered [ 9264.107491] LNet: 269054:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9264.128901] LNetError: 269054:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9264.165881] LNet: Removed LNI 192.168.201.119@tcp [ 9265.254374] Key type .llcrypt unregistered [ 9265.257546] Key type ._llcrypt unregistered [ 9296.220976] Key type ._llcrypt registered [ 9296.222609] Key type .llcrypt registered [ 9296.424670] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 9297.754050] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9297.930482] alg: No test for adler32 (adler32-zlib) [ 9299.147375] Lustre: Lustre: Build Version: 2.17.57_113_gf5ca795 [ 9299.419946] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 9301.176324] Key type lgssc registered [ 9302.552632] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9312.421585] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9312.474354] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9314.040067] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9319.091504] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 9324.525506] LustreError: 270518:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9324.540893] LustreError: 270518:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 9328.957684] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9329.504363] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9335.246327] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 9343.743369] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9350.170469] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9358.988349] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9359.330054] LustreError: 272337:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9359.545628] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9364.998923] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 9367.867993] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 9376.085422] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9377.035078] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 9379.299049] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9379.307251] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9379.332364] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9379.352160] Lustre: Skipped 1 previous similar message [ 9381.388469] Lustre: server umount lustre-OST0000 complete [ 9389.542682] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9389.550675] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9389.554230] Lustre: Skipped 1 previous similar message [ 9389.565578] Lustre: Skipped 2 previous similar messages [ 9394.658760] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9399.783898] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9399.797734] Lustre: Skipped 2 previous similar messages [ 9401.313984] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9401.751691] Lustre: server umount lustre-MDT0000 complete [ 9404.909760] LustreError: 270518:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9404.932305] LustreError: 270518:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9405.456674] LustreError: 273046:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788846114 with bad export cookie 6350090416874550597 [ 9405.459571] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9405.468205] LustreError: 273046:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9405.844678] Lustre: server umount lustre-MDT0001 complete [ 9418.726634] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 9421.969500] Key type lgssc unregistered [ 9422.289180] LNet: 274103:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9422.295728] LNetError: 274103:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9422.320948] LNet: Removed LNI 192.168.201.119@tcp [ 9423.390291] Key type .llcrypt unregistered [ 9423.399103] Key type ._llcrypt unregistered [ 9441.549723] Key type ._llcrypt registered [ 9441.554850] Key type .llcrypt registered [ 9441.694974] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 9442.945129] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9442.968523] alg: No test for adler32 (adler32-zlib) [ 9444.020815] Lustre: Lustre: Build Version: 2.17.57_113_gf5ca795 [ 9444.391489] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 9446.216599] Key type lgssc registered [ 9447.740752] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9457.325732] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9457.357186] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9472.928443] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 9478.049126] LustreError: 275294:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.119@tcp: failed processing log, type 4: rc = -110 [ 9504.736262] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9512.053095] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 9520.821704] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9523.833563] Lustre: server umount lustre-OST0000 complete [ 9541.670889] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [ 9544.045892] Key type lgssc unregistered [ 9544.284986] LNet: 276924:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9544.290854] LNetError: 276924:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9545.329174] LNet: Removed LNI 192.168.201.119@tcp [ 9546.437203] Key type .llcrypt unregistered [ 9546.440304] Key type ._llcrypt unregistered [ 9595.488970] Key type ._llcrypt registered [ 9595.493081] Key type .llcrypt registered [ 9595.601908] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 9597.129389] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9597.289050] alg: No test for adler32 (adler32-zlib) [ 9598.450251] Lustre: Lustre: Build Version: 2.17.57_113_gf5ca795 [ 9598.795257] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [ 9600.592536] Key type lgssc registered [ 9602.183855] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9619.766675] loop0: detected capacity change from 0 to 204800000 [ 9620.023393] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9620.040265] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9620.499262] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9621.803175] loop0: detected capacity change from 0 to 204800000 [ 9622.091017] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9622.111686] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9622.659833] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9624.066137] loop0: detected capacity change from 0 to 204800000 [ 9624.309752] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9624.321643] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9624.476910] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9626.165100] loop0: detected capacity change from 0 to 204800000 [ 9626.438197] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9626.454438] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9626.574078] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9628.074493] loop0: detected capacity change from 0 to 204800000 [ 9628.145103] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9633.201672] loop0: detected capacity change from 0 to 204800000 [ 9633.233133] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9638.819547] loop0: detected capacity change from 0 to 204800000 [ 9638.866322] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9644.184613] loop0: detected capacity change from 0 to 204800000 [ 9644.232873] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9649.551515] loop0: detected capacity change from 0 to 204800000 [ 9649.690639] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9649.773031] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9649.901848] Lustre: 280339:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9649.916152] Lustre: 280339:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 9649.930103] Lustre: 280339:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 9649.955926] Lustre: 280339:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9649.964571] Lustre: 280339:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9649.973715] Lustre: 280339:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9649.982836] Lustre: 280339:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 9649.990882] Lustre: 280339:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 9649.998241] Lustre: 280339:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 9650.018456] Lustre: 280339:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 9650.036310] Lustre: 280339:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 9650.049671] Lustre: 280339:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 9650.068165] Lustre: 280339:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 9650.079405] Lustre: 280339:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 9656.797825] Lustre: server umount MGS complete [ 9658.009373] loop0: detected capacity change from 0 to 204800000 [ 9658.060753] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9658.838472] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9658.912350] Lustre: lustre-MDT0000: Aborting client recovery [ 9660.079404] loop1: detected capacity change from 0 to 204800000 [ 9660.192915] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9660.337381] Lustre: 280775:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9660.354942] Lustre: 280775:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 9660.369726] Lustre: 280775:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9660.393350] Lustre: 280775:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 9660.407478] Lustre: 280775:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 9660.414362] Lustre: 280775:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9660.422429] Lustre: 280775:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9660.457700] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9660.681269] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9660.770395] Lustre: lustre-MDT0001: Aborting client recovery [ 9661.924885] loop2: detected capacity change from 0 to 204800000 [ 9662.227653] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9662.344596] Lustre: 280855:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9662.358247] Lustre: 280855:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9662.367441] Lustre: 280855:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9662.376187] Lustre: 280855:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9662.385342] Lustre: 280855:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9662.396470] Lustre: 280855:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9662.404250] Lustre: 280855:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9662.491219] Lustre: Found index 0 for lustre-OST0000, updating log [ 9662.513609] Lustre: 280873:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9662.578443] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9662.600256] Lustre: lustre-OST0000: Aborting recovery [ 9663.504535] loop3: detected capacity change from 0 to 204800000 [ 9663.762032] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9663.859862] Lustre: 280938:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9663.886353] Lustre: 280938:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9663.909797] Lustre: 280938:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9663.928607] Lustre: 280938:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9663.944861] Lustre: 280938:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9663.964348] Lustre: 280938:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9663.980471] Lustre: 280938:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9663.999312] Lustre: Found index 1 for lustre-OST0001, updating log [ 9664.035317] Lustre: 280947:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9664.112255] Lustre: lustre-OST0001: Aborting recovery [ 9668.217789] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9668.230119] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9668.274449] LustreError: 280718:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 9668.320583] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 9668.322572] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 9670.252758] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 9670.254956] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 9674.798836] LustreError: 280685:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 871us: rc = -2 [ 9675.366383] LustreError: 280686:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 532us: rc = -2 [ 9675.386159] LustreError: 280686:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 22 previous similar messages [ 9691.108174] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9691.111280] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9691.124712] Lustre: Skipped 2 previous similar messages [ 9691.135817] Lustre: Skipped 3 previous similar messages [ 9696.228872] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9696.238121] Lustre: Skipped 3 previous similar messages [ 9701.346173] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9701.359163] Lustre: Skipped 3 previous similar messages [ 9702.368160] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9702.670133] Lustre: server umount lustre-MDT0000 complete [ 9704.032526] LustreError: 280668:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788846413 with bad export cookie 17645236505972421649 [ 9704.050550] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9704.080278] LustreError: 280668:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9706.467251] LustreError: 280683:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9706.469270] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9706.469856] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9706.482571] LustreError: 280683:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9706.503510] Lustre: Skipped 2 previous similar messages [ 9709.474799] LustreError: 280682:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9709.495963] LustreError: 280682:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9710.894180] Lustre: server umount lustre-MDT0001 complete [ 9718.248829] Lustre: server umount lustre-OST0000 complete [ 9725.921347] Lustre: server umount lustre-OST0001 complete [ 9736.520891] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 01:47:23 (1788846443) [ 9769.015440] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_hostid [ 9780.203469] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [ 9823.348944] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9823.673987] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9823.728609] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9823.931846] Lustre: lustre-MDT0000: new disk, initializing [ 9824.100509] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9824.110450] Lustre: Skipped 1 previous similar message [ 9824.163863] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9830.359983] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 9843.353710] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9843.459628] Lustre: 285023:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9843.500982] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9843.503576] Lustre: Skipped 1 previous similar message [ 9843.545582] Lustre: lustre-MDT0001: new disk, initializing [ 9843.618925] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9843.654215] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9843.676084] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9848.129390] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 9859.381069] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9866.402709] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9874.789446] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9875.357450] Lustre: lustre-OST0000: new disk, initializing [ 9875.362550] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9875.370859] Lustre: 286998:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9875.507927] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9881.638464] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [ 9884.245315] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9884.253523] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9884.345654] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9891.508813] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9897.596327] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9908.724708] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9913.661903] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9919.925774] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9971.684347] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9971.694715] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9971.715420] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9971.726098] Lustre: Skipped 4 previous similar messages [ 9973.224849] Lustre: server umount lustre-OST0000 complete [ 9981.921488] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9981.934721] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9981.935255] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9981.940979] Lustre: Skipped 2 previous similar messages [ 9981.958146] Lustre: Skipped 1 previous similar message [ 9983.532596] Lustre: server umount lustre-MDT0000 complete [ 9987.042215] LustreError: 285036:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9988.052209] LustreError: 285016:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788846697 with bad export cookie 17645236505972424456 [ 9988.057078] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9988.067728] LustreError: 285016:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9988.808914] Lustre: server umount lustre-MDT0001 complete [ 9999.724845] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10005.669169] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [10011.608973] Lustre: server umount MGS complete [10026.123312] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10026.690651] LustreError: 290772:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10026.722045] LustreError: 290772:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10026.729600] LustreError: 290785:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10026.815614] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10032.102048] LustreError: 290773:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10032.159268] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [10041.315144] LustreError: 290773:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10041.332440] LustreError: 290773:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10043.456329] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10049.940461] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [10059.411785] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10065.609485] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10073.108731] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10073.578664] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10073.583908] Lustre: Skipped 1 previous similar message [10081.344829] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [10090.666442] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10126.839440] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10126.853074] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10126.868097] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10126.871470] Lustre: Skipped 1 previous similar message [10126.902609] Lustre: Skipped 1 previous similar message [10140.129668] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10140.317089] Lustre: server umount lustre-OST0000 complete [10146.273639] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10146.294643] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10150.808542] Lustre: server umount lustre-MDT0000 complete [10152.418314] LustreError: 290773:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10152.442106] LustreError: 290773:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10154.755251] LustreError: 290754:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788846863 with bad export cookie 17645236505972426395 [10154.763532] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10154.767597] LustreError: 290754:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10155.222595] Lustre: server umount lustre-MDT0001 complete [10169.473913] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [10172.955709] Key type lgssc unregistered [10173.292339] LNet: 294719:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10173.298652] LNetError: 294719:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10173.328615] LNet: Removed LNI 192.168.201.119@tcp [10174.249302] Key type .llcrypt unregistered [10174.252491] Key type ._llcrypt unregistered [10222.495948] Key type ._llcrypt registered [10222.501255] Key type .llcrypt registered [10222.630647] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_hostid [10240.400643] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [10241.702423] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10241.731579] alg: No test for adler32 (adler32-zlib) [10243.131381] Lustre: Lustre: Build Version: 2.17.57_113_gf5ca795 [10243.526973] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [10245.296216] Key type lgssc registered [10246.860936] Lustre: Echo OBD driver; http://www.lustre.org/ [10287.007444] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10287.052865] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10288.307522] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10288.345927] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10288.437324] Lustre: lustre-MDT0000: new disk, initializing [10288.568497] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10288.593364] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10293.705875] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [10305.646375] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10305.775914] Lustre: 298675:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10305.886308] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10305.891019] Lustre: Skipped 1 previous similar message [10305.955288] Lustre: lustre-MDT0001: new disk, initializing [10306.045611] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10306.083680] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10306.093665] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10311.048458] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [10322.565636] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10329.177880] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10336.401397] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10336.650488] Lustre: lustre-OST0000: new disk, initializing [10336.657039] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10336.663923] Lustre: 300647:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10336.798924] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10343.653879] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [10345.539405] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10345.548407] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10345.670775] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10356.332749] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10365.770520] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10379.910059] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10386.145516] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10396.659971] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [10438.122114] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10438.122529] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10438.126711] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10438.149115] Lustre: Skipped 1 previous similar message [10438.179140] LustreError: Skipped 1 previous similar message [10441.721503] Lustre: server umount lustre-OST0000 complete [10448.354736] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10448.376161] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10448.380729] Lustre: Skipped 1 previous similar message [10452.907308] Lustre: server umount lustre-MDT0000 complete [10453.480426] LustreError: 299739:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10453.510343] LustreError: 299739:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10456.775873] LustreError: 301900:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788847165 with bad export cookie 11044865998863971086 [10456.792493] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10457.123146] Lustre: server umount lustre-MDT0001 complete [10465.899244] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10471.827834] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [10477.275935] Lustre: server umount MGS complete [10489.502762] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10489.997696] LustreError: 304522:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10490.049156] LustreError: 304535:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10490.115739] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10495.381914] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [10495.461509] LustreError: 304523:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10495.497628] LustreError: 304523:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10500.579364] LustreError: 304522:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10505.706262] LustreError: 304523:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10505.860659] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10506.271081] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10511.198416] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [10519.758196] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10527.595375] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10536.363165] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10536.666254] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10546.314951] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [10557.222481] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10597.354249] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10597.358225] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10597.373265] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10597.382777] Lustre: Skipped 2 previous similar messages [10597.417220] Lustre: Skipped 1 previous similar message [10599.912507] Lustre: server umount lustre-OST0000 complete [10607.588217] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10607.607875] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10611.034164] Lustre: server umount lustre-MDT0000 complete [10612.705737] LustreError: 304517:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10612.728442] LustreError: 304517:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10615.228284] LustreError: 304502:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788847324 with bad export cookie 11044865998863973039 [10615.230632] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10615.241632] LustreError: 304502:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [10615.647980] Lustre: server umount lustre-MDT0001 complete [10628.023819] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [10631.039187] Key type lgssc unregistered [10631.394619] LNet: 308481:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10631.405179] LNetError: 308481:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10631.427669] LNet: Removed LNI 192.168.201.119@tcp [10632.417172] Key type .llcrypt unregistered [10632.421458] Key type ._llcrypt unregistered [10654.767169] Key type ._llcrypt registered [10654.769300] Key type .llcrypt registered [10654.892888] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_hostid [10669.923844] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [10671.329593] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10671.346056] alg: No test for adler32 (adler32-zlib) [10672.492317] Lustre: Lustre: Build Version: 2.17.57_113_gf5ca795 [10672.810462] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [10674.496448] Key type lgssc registered [10675.820055] Lustre: Echo OBD driver; http://www.lustre.org/ [10704.812124] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [10722.056238] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10722.090426] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10723.321040] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10723.357796] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10723.450660] Lustre: lustre-MDT0000: new disk, initializing [10723.522412] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10723.539490] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10728.268176] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [10740.867976] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10741.219897] Lustre: lustre-OST0000: new disk, initializing [10741.223585] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10741.230580] Lustre: Skipped 1 previous similar message [10741.235919] Lustre: 312530:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10741.328531] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10747.545050] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [10747.555785] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [10747.608312] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [10748.257717] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [11764.731209] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11764.755988] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11766.447697] Lustre: server umount lustre-MDT0000 complete [11770.940505] LustreError: 311717:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788848480 with bad export cookie 17234981815033966068 [11770.947191] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11770.952183] LustreError: 311717:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11771.145571] Lustre: server umount lustre-OST0000 complete [11805.162623] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_hostid [11819.123736] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [11864.826217] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11865.172968] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11865.220982] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11865.324807] Lustre: lustre-MDT0000: new disk, initializing [11865.425130] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11865.444936] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11870.381321] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [11881.674841] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11881.825260] Lustre: 316897:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11881.879716] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11881.885233] Lustre: Skipped 1 previous similar message [11881.968504] Lustre: lustre-MDT0001: new disk, initializing [11882.119688] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11882.157791] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11882.174535] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11887.571428] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [11898.081401] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11905.325630] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11913.358828] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11913.757369] Lustre: lustre-OST0000: new disk, initializing [11913.765550] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11913.779365] Lustre: 318865:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11913.907586] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11920.006654] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11920.032328] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11920.229317] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11921.705755] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [11935.969397] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11941.803149] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11942.083318] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11947.758455] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11948.176585] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11951.074664] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11951.084701] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11951.132428] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11956.198963] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11956.208175] Lustre: Skipped 1 previous similar message [11956.771553] Lustre: server umount lustre-OST0000 complete [11963.873567] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11963.883188] LustreError: Skipped 1 previous similar message [11963.894316] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11963.911059] Lustre: Skipped 1 previous similar message [11963.922941] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11966.442094] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11969.957331] Lustre: server umount lustre-MDT0000 complete [11975.516088] LustreError: 316889:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788848684 with bad export cookie 17234981815036906887 [11975.520509] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11975.526729] LustreError: 316889:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11976.329142] Lustre: server umount lustre-MDT0001 complete [11981.139806] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [11983.420351] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 02:24:50 (1788848690) [11992.147883] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11992.708891] LustreError: 321180:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11992.727215] LustreError: 321180:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11992.838288] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11998.178245] LustreError: 321181:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11998.414871] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [12002.275869] LustreError: 321180:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12007.393711] LustreError: 321181:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12008.437664] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12008.841886] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12014.980911] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [12024.687521] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12031.554179] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12040.319460] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12040.894330] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12048.653403] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [12057.661558] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12065.716709] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12065.947897] Lustre: lustre-OST0001: new disk, initializing [12065.956669] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [12065.973163] Lustre: 324018:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12066.187983] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12074.406061] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [12074.631761] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [12074.638652] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [12074.710918] LustreError: 324385:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12075.756943] LustreError: 324385:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12075.773430] LustreError: 324385:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [12076.844187] LustreError: 324385:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12076.868507] LustreError: 324385:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [12078.887258] LustreError: 324394:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [12078.912391] LustreError: 324394:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [12083.045343] LustreError: 324385:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12083.058605] LustreError: 324385:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [12087.477681] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12087.730926] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12091.378068] LustreError: 324385:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12091.390176] LustreError: 324385:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 16 previous similar messages [12095.285974] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff94bd50a01000.ost_server_uuid 50 [12097.119343] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff94bd50a01000.ost_server_uuid in FULL state after 0 sec [12104.285988] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff94bd50a01000.ost_server_uuid 50 [12106.403711] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff94bd50a01000.ost_server_uuid in IDLE state after 0 sec [12107.943448] LustreError: 324385:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12107.968902] LustreError: 324385:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 31 previous similar messages [12108.980368] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [12124.643125] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12124.643144] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12124.650589] LustreError: Skipped 1 previous similar message [12124.657382] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12124.669289] Lustre: Skipped 1 previous similar message [12124.688375] Lustre: Skipped 3 previous similar messages [12129.350238] Lustre: server umount lustre-OST0001 complete [12131.297442] LustreError: 325446:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12131.312246] LustreError: 325446:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12139.490900] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12139.511612] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12139.528476] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12139.543250] Lustre: Skipped 3 previous similar messages [12141.541069] LustreError: 325446:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12141.560508] LustreError: 325446:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [12144.139934] Lustre: server umount lustre-OST0000 complete [12155.772333] Lustre: server umount lustre-MDT0000 complete [12160.233473] LustreError: 321162:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788848869 with bad export cookie 17234981815036907769 [12160.240845] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12160.256452] LustreError: 321162:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12160.684233] Lustre: server umount lustre-MDT0001 complete [12174.802595] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [12178.512439] Key type lgssc unregistered [12178.884837] LNet: 326710:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12178.898299] LNetError: 326710:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12178.931480] LNet: Removed LNI 192.168.201.119@tcp [12179.999472] Key type .llcrypt unregistered [12180.010210] Key type ._llcrypt unregistered [12208.667591] Key type ._llcrypt registered [12208.670858] Key type .llcrypt registered [12208.794275] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [12210.550326] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12210.592063] alg: No test for adler32 (adler32-zlib) [12211.585759] Lustre: Lustre: Build Version: 2.17.57_113_gf5ca795 [12211.784801] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [12213.456330] Key type lgssc registered [12214.709729] Lustre: Echo OBD driver; http://www.lustre.org/ [12222.492235] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12222.535821] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12223.997291] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12228.576641] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [12234.216124] LustreError: 328126:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12234.250629] LustreError: 328126:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12236.525364] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12239.335093] LustreError: 328125:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12243.715255] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12244.067497] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12249.998967] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [12261.211672] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12261.717075] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12268.905900] LustreError: 329804:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12268.969281] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [12269.962332] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [12275.177589] LustreError: 329804:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12275.206957] LustreError: 329804:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12276.952111] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12280.293361] LustreError: 329807:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12280.318135] LustreError: 329807:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12285.068681] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12285.478575] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12290.586959] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [12292.697681] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [12301.681521] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12310.319034] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [12310.569468] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12418.542257] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12418.545906] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12418.570818] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12418.588675] LustreError: Skipped 1 previous similar message [12423.659368] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12423.667365] Lustre: Skipped 2 previous similar messages [12428.777678] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12429.280136] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12429.594615] Lustre: server umount lustre-OST0001 complete [12433.895945] LustreError: 329806:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12433.919851] LustreError: 329806:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12437.986852] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12438.002742] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12438.019673] Lustre: Skipped 1 previous similar message [12438.044718] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12438.054190] Lustre: Skipped 1 previous similar message [12439.024046] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12439.024423] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12444.102740] Lustre: server umount lustre-OST0000 complete [12448.737910] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12448.742731] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12448.748979] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12448.751712] Lustre: Skipped 2 previous similar messages [12454.505935] Lustre: server umount lustre-MDT0000 complete [12458.122727] LustreError: 328107:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788849167 with bad export cookie 9266481032384052137 [12458.123803] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12458.131101] LustreError: 328107:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12458.938979] Lustre: server umount lustre-MDT0001 complete [12471.685733] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [12474.657988] Key type lgssc unregistered [12475.136988] LNet: 333071:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12475.150272] LNetError: 333071:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12475.178547] LNet: Removed LNI 192.168.201.119@tcp [12476.534275] Key type .llcrypt unregistered [12476.536277] Key type ._llcrypt unregistered [12523.158844] Key type ._llcrypt registered [12523.163044] Key type .llcrypt registered [12523.280379] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [12524.537977] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12524.572828] alg: No test for adler32 (adler32-zlib) [12525.696210] Lustre: Lustre: Build Version: 2.17.57_113_gf5ca795 [12525.935827] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [12527.584242] Key type lgssc registered [12528.747055] Lustre: Echo OBD driver; http://www.lustre.org/ [12538.572596] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12538.590397] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12540.096359] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12545.254557] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [12545.526195] LustreError: 335358:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12545.551298] LustreError: 335358:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12550.627179] LustreError: 335359:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12551.658840] LustreError: 335358:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12554.379545] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12554.839170] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12559.041294] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [12561.513604] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [12567.346879] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12567.630968] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12573.712596] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [12579.887209] LustreError: 336844:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12582.271718] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12582.545134] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12586.713187] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [12586.714908] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [12589.757055] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [12598.791638] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12608.819689] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [12621.283639] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12621.293835] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12621.315834] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12622.825743] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12622.836928] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12622.841960] Lustre: Skipped 3 previous similar messages [12626.835702] Lustre: server umount lustre-MDT0000 complete [12627.952987] LustreError: 335354:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12627.992035] LustreError: 335354:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12630.732857] LustreError: 335338:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788849339 with bad export cookie 13884856276093627230 [12630.749393] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12630.754952] LustreError: 335338:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [12631.378475] Lustre: server umount lustre-MDT0001 complete [12646.003310] Lustre: server umount lustre-OST0000 complete [12648.928878] Lustre: 334620:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788849342/real 1788849342] req@ffff8e041596fb80 x1875744375504256/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788849358 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [12648.969698] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12649.004467] Lustre: Skipped 2 previous similar messages [12650.200486] Lustre: server umount lustre-OST0001 complete [12665.319118] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [12676.336722] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12676.737103] LustreError: 340623:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12676.797874] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12681.272727] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [12690.235212] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12690.542971] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12695.488539] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [12698.345268] Lustre: 341763:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [12704.762765] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12711.478306] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [12715.381546] LustreError: 342118:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12715.395982] LustreError: 342118:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12721.004370] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12721.306603] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12721.312064] Lustre: Skipped 1 previous similar message [12722.422454] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [12722.434275] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [12727.931743] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [12735.386765] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12739.199679] Lustre: 343634:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [12746.721280] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12746.737453] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12746.756444] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12749.893545] Lustre: server umount lustre-MDT0000 complete [12753.381826] LustreError: 340618:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12753.406210] LustreError: 340618:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [12754.271856] LustreError: 340604:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788849463 with bad export cookie 13884856276093628931 [12754.280953] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12754.290633] LustreError: 340604:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12754.631981] Lustre: server umount lustre-MDT0001 complete [12768.218221] Lustre: server umount lustre-OST0000 complete [12783.115551] Lustre: server umount lustre-OST0001 complete [12787.085040] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [12788.664898] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [12790.358493] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 02:38:17 (1788849497) [12812.210322] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [12821.049195] loop0: detected capacity change from 0 to 1099511627776 [12821.683249] SGI XFS with ACLs, security attributes, quota, no debug enabled [12821.692910] *** VALIDATE xfs *** [12821.718300] XFS (loop0): Mounting V5 Filesystem [12821.753868] XFS (loop0): Ending clean mount [12823.024613] loop1: detected capacity change from 0 to 18691697672192 [12928.715660] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_hostid [12937.588887] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [12972.204159] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12972.491250] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12972.550396] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12972.657784] Lustre: lustre-MDT0000: new disk, initializing [12972.777739] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12972.806476] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12977.405447] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [12988.671207] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12988.801213] Lustre: 348772:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12988.867847] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12988.872544] Lustre: Skipped 1 previous similar message [12988.912390] Lustre: lustre-MDT0001: new disk, initializing [12989.030824] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12989.039688] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12993.673524] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [13003.062888] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13008.006592] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13013.588461] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13013.798223] Lustre: lustre-OST0000: new disk, initializing [13013.802126] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13013.807823] Lustre: 350740:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13015.492330] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13015.504192] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13015.607083] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13019.050672] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [13028.280424] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13033.606567] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [13033.828184] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [13038.619728] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [13038.831971] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [13041.123483] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [13041.133867] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13041.136227] LustreError: Skipped 1 previous similar message [13041.139257] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13041.143658] Lustre: Skipped 4 previous similar messages [13041.156663] Lustre: Skipped 5 previous similar messages [13046.250686] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13046.262914] Lustre: Skipped 1 previous similar message [13046.697729] Lustre: server umount lustre-OST0000 complete [13051.364650] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13056.482379] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13056.492457] Lustre: Skipped 3 previous similar messages [13056.670760] Lustre: server umount lustre-MDT0000 complete [13060.206488] LustreError: 349578:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788849769 with bad export cookie 13884856276093630485 [13060.208476] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13060.216509] LustreError: 349578:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [13060.497937] Lustre: server umount lustre-MDT0001 complete [13067.886889] XFS (loop0): Unmounting Filesystem [13072.934508] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 02:43:00 (1788849780) [13079.843903] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13080.248837] LustreError: 353299:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13080.333214] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13080.337724] Lustre: Skipped 2 previous similar messages [13085.031939] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [13093.924828] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13097.740710] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [13104.562244] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13110.147901] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13116.618379] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13123.996628] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [13131.730222] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13139.435853] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [13139.444838] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13139.457129] Lustre: Skipped 1 previous similar message [13139.462986] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13153.250190] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [13153.434200] Lustre: server umount lustre-OST0000 complete [13160.941756] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13162.926595] Lustre: server umount lustre-MDT0000 complete [13166.455442] LustreError: 353279:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788849875 with bad export cookie 13884856276093631367 [13166.455823] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13166.463565] LustreError: 353279:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [13166.772465] Lustre: server umount lustre-MDT0001 complete [13177.305605] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [13179.851566] Key type lgssc unregistered [13180.067858] LNet: 356932:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13180.073395] LNetError: 356932:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13181.096114] LNet: Removed LNI 192.168.201.119@tcp [13181.974165] Key type .llcrypt unregistered [13181.975888] Key type ._llcrypt unregistered [13206.865313] Key type ._llcrypt registered [13206.866925] Key type .llcrypt registered [13206.938728] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [13207.795733] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [13207.894106] alg: No test for adler32 (adler32-zlib) [13209.116986] Lustre: Lustre: Build Version: 2.17.57_113_gf5ca795 [13209.427736] LNet: Added LNI 192.168.201.119@tcp [8/256/0/180] [13211.144761] Key type lgssc registered [13212.098717] Lustre: Echo OBD driver; http://www.lustre.org/ [13219.725770] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [13219.753781] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13221.185794] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13225.772871] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [13231.595319] LustreError: 358493:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13231.611065] LustreError: 358493:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [13234.001898] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13234.308674] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13238.265868] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [13245.176323] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13250.217672] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13256.972021] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13257.457903] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13263.002382] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [13269.397335] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13277.154730] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13277.182407] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13282.282850] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13282.288470] Lustre: Skipped 3 previous similar messages [13282.446129] Lustre: server umount lustre-MDT0001 complete [13287.400114] LustreError: 358494:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13287.426218] LustreError: 358494:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [13291.571510] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13291.646286] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13291.651443] Lustre: Found index 1 for lustre-MDT0001, updating log [13291.690186] Lustre: 358480:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13292.018845] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13295.588169] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [13297.136984] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13298.690235] LustreError: 362093:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13298.698556] LustreError: 362093:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13298.708474] LustreError: 358486:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13298.758430] LustreError: 362096:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.119@tcp: cfg command failed: rc = -17 [13298.762155] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [13298.762155] [13604.337303] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13604.346742] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13604.354808] Lustre: Skipped 1 previous similar message [13604.367499] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13604.371942] Lustre: Skipped 2 previous similar messages [13606.887585] Lustre: server umount lustre-MDT0001 complete [13609.441892] LustreError: 359555:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13614.567662] LustreError: 358489:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13614.581958] LustreError: 358489:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [13615.076717] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13615.153699] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13615.156804] Lustre: Found index 1 for lustre-MDT0001, updating log [13615.184948] Lustre: 358480:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13615.424638] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13619.472647] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [13620.729778] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13621.247302] LustreError: 371788:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13621.258083] LustreError: 371788:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13621.270085] LustreError: 358486:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13621.275426] LustreError: 358486:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13621.349498] LustreError: 371797:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [13621.358872] LustreError: 371797:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [13621.366031] LustreError: 371797:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.119@tcp: cfg command failed: rc = -17 [13621.379105] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.201.119@tcp [13621.379105] [13927.425625] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.19@tcp (stopping) [13927.428414] Lustre: Skipped 1 previous similar message [13927.909222] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13927.919849] Lustre: Skipped 2 previous similar messages [13927.924268] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13929.504722] Lustre: server umount lustre-MDT0001 complete [13932.527245] LustreError: 364983:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.19@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13932.548886] LustreError: 364983:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [13936.442634] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13936.509960] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13936.512391] Lustre: Found index 1 for lustre-MDT0001, updating log [13936.536481] Lustre: 358480:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13936.743065] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13939.295196] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [13942.249936] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13943.793901] LustreError: 381751:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13943.802590] LustreError: 381751:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13943.813186] LustreError: 358486:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13943.827599] LustreError: 358486:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13943.875713] LustreError: 381754:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [13943.882652] LustreError: 381754:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.119@tcp: cfg command failed: rc = -114 [13943.887426] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [13943.887426] [14259.171782] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [14259.180489] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14259.200673] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14259.214650] Lustre: Skipped 2 previous similar messages [14265.459139] Lustre: server umount lustre-MDT0000 complete [14269.620202] LustreError: 361679:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788850978 with bad export cookie 4832554592527152685 [14269.624137] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14269.633803] LustreError: 361679:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [14270.434984] LustreError: 364983:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14270.436071] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14270.458117] LustreError: 364983:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [14270.470097] Lustre: Skipped 2 previous similar messages [14270.476128] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [14270.481536] Lustre: Skipped 6 previous similar messages [14276.280411] Lustre: server umount lustre-MDT0001 complete [14280.690520] Lustre: server umount lustre-OST0000 complete [14290.537869] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_hostid [14301.638615] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [14352.731813] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 03:04:19 (1788851059) [14359.642952] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14359.920532] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [14359.955195] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [14360.046518] Lustre: lustre-MDT0000: new disk, initializing [14360.187785] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [14360.231949] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [14366.276320] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [14379.104284] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14379.253588] Lustre: 391885:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [14379.305349] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [14379.314272] Lustre: Skipped 1 previous similar message [14379.446516] Lustre: lustre-MDT0001: new disk, initializing [14379.585679] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [14379.658339] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [14379.675958] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [14384.991783] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [14396.138464] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [14403.730317] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [14412.716289] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14413.122362] Lustre: lustre-OST0000: new disk, initializing [14413.129674] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [14413.148621] Lustre: 393856:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [14413.296840] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [14421.423131] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_default_debug -1 all [14423.123148] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [14423.137475] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [14423.317336] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [14433.383864] Lustre: DEBUG MARKER: oleg119-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [14441.709738] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [14441.720560] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14441.729063] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14441.731361] Lustre: Skipped 1 previous similar message [14442.111527] Lustre: server umount lustre-MDT0000 complete [14444.002292] LustreError: 391896:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14444.027227] LustreError: 391896:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [14446.932347] LustreError: 391877:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788851156 with bad export cookie 4832554592527154638 [14446.938377] LustreError: 391877:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [14446.941934] LustreError: MGC192.168.201.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14447.410059] Lustre: server umount lustre-MDT0001 complete [14462.843343] Lustre: server umount lustre-OST0000 complete [14499.291414] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing set_hostid [14511.952088] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing load_modules_local [14557.919930] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 14291 sec ========= 03:07:44 (1788851264) [14560.348904] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 03:07:47 (1788851267) === [14564.266581] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 03:07:51 (1788851271) === [14612.565937] Lustre: DEBUG MARKER: oleg119-server.virtnet: executing unload_modules_local [14615.993636] Key type lgssc unregistered [14616.260880] LNet: 400227:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [14616.267049] LNetError: 400227:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [14616.277533] LNet: Removed LNI 192.168.201.119@tcp [14617.345436] Key type .llcrypt unregistered [14617.347374] Key type ._llcrypt unregistered