[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 631190787 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001009] APIC: Switch to symmetric I/O mode setup [ 0.002349] x2apic enabled [ 0.003010] Switched APIC routing to physical x2apic. [ 0.005009] kvm-guest: setup PV IPIs [ 0.008855] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009019] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010012] pid_max: default: 32768 minimum: 301 [ 0.012022] LSM: Security Framework initializing [ 0.013062] Yama: becoming mindful. [ 0.014049] SELinux: Initializing. [ 0.015073] *** VALIDATE selinux *** [ 0.023608] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.028478] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.029148] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030142] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031116] *** VALIDATE tmpfs *** [ 0.033433] *** VALIDATE proc *** [ 0.034262] *** VALIDATE cgroup *** [ 0.035011] *** VALIDATE cgroup2 *** [ 0.037120] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.038148] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.039008] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.041020] Spectre V2 : User space: Vulnerable [ 0.042008] Speculative Store Bypass: Vulnerable [ 0.044829] debug: unmapping init [mem 0xffffffff99259000-0xffffffff99260fff] [ 0.046215] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.047679] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.048025] ... version: 2 [ 0.049016] ... bit width: 48 [ 0.050015] ... generic registers: 4 [ 0.051015] ... value mask: 0000ffffffffffff [ 0.052016] ... max period: 00007fffffffffff [ 0.053017] ... fixed-purpose events: 3 [ 0.054012] ... event mask: 000000070000000f [ 0.055265] rcu: Hierarchical SRCU implementation. [ 0.057490] smp: Bringing up secondary CPUs ... [ 0.058586] x86: Booting SMP configuration: [ 0.059042] .... node #0, CPUs: #1 #2 #3 [ 0.063596] smp: Brought up 1 node, 4 CPUs [ 0.065023] smpboot: Max logical packages: 1 [ 0.066016] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.113071] node 0 deferred pages initialised in 45ms [ 0.118107] devtmpfs: initialized [ 0.119257] x86/mm: Memory block size: 128MB [ 0.123118] gcov: version magic: 0x41383552 [ 0.125333] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.126081] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.127356] pinctrl core: initialized pinctrl subsystem [ 0.128214] [ 0.129010] ************************************************************* [ 0.130015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.131023] ** ** [ 0.132016] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.133015] ** ** [ 0.134014] ** This means that this kernel is built to expose internal ** [ 0.135016] ** IOMMU data structures, which may compromise security on ** [ 0.136016] ** your system. ** [ 0.137014] ** ** [ 0.138012] ** If you see this message and you are not debugging the ** [ 0.139015] ** kernel, report this immediately to your vendor! ** [ 0.140015] ** ** [ 0.141015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.142018] ************************************************************* [ 0.143793] NET: Registered protocol family 16 [ 0.144540] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.145063] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.146064] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.147490] cpuidle: using governor menu [ 0.149933] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.152611] PCI: Using configuration type 1 for base access [ 0.154124] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.164088] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.166038] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.169072] cryptd: max_cpu_qlen set to 1000 [ 0.170355] ACPI: Added _OSI(Module Device) [ 0.172015] ACPI: Added _OSI(Processor Device) [ 0.174011] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.175012] ACPI: Added _OSI(Processor Aggregator Device) [ 0.180866] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.188209] ACPI: Interpreter enabled [ 0.189062] ACPI: PM: (supports S0 S3 S4 S5) [ 0.190013] ACPI: Using IOAPIC for interrupt routing [ 0.191098] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.192382] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.203633] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.204043] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.205018] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.206077] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.208584] acpiphp: Slot [2] registered [ 0.209180] acpiphp: Slot [5] registered [ 0.210115] acpiphp: Slot [6] registered [ 0.211112] acpiphp: Slot [7] registered [ 0.212170] acpiphp: Slot [8] registered [ 0.213105] acpiphp: Slot [9] registered [ 0.214216] acpiphp: Slot [10] registered [ 0.215088] acpiphp: Slot [3] registered [ 0.216230] acpiphp: Slot [4] registered [ 0.217140] acpiphp: Slot [11] registered [ 0.218106] acpiphp: Slot [12] registered [ 0.219109] acpiphp: Slot [13] registered [ 0.220000] acpiphp: Slot [14] registered [ 0.222099] acpiphp: Slot [15] registered [ 0.224228] acpiphp: Slot [16] registered [ 0.226167] acpiphp: Slot [17] registered [ 0.227241] acpiphp: Slot [18] registered [ 0.229143] acpiphp: Slot [19] registered [ 0.231104] acpiphp: Slot [20] registered [ 0.233109] acpiphp: Slot [21] registered [ 0.235143] acpiphp: Slot [22] registered [ 0.237111] acpiphp: Slot [23] registered [ 0.238105] acpiphp: Slot [24] registered [ 0.240099] acpiphp: Slot [25] registered [ 0.242102] acpiphp: Slot [26] registered [ 0.244102] acpiphp: Slot [27] registered [ 0.246122] acpiphp: Slot [28] registered [ 0.247097] acpiphp: Slot [29] registered [ 0.249104] acpiphp: Slot [30] registered [ 0.251153] acpiphp: Slot [31] registered [ 0.253126] PCI host bridge to bus 0000:00 [ 0.255017] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.257021] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.260020] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.262020] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.265022] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.268022] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.270165] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.273201] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.276701] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.287961] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.294203] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.297025] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.300018] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.304021] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.306827] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.310139] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.313050] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.316962] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.323016] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.339141] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.346016] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.352689] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.360015] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.369014] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.388017] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.397140] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.411015] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.418014] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.427013] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.442452] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.454015] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.467014] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.487017] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.502139] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.510017] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.517014] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.539015] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.553652] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.563016] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.576016] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.616015] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.629207] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.644026] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.654095] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.681028] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.694000] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.697373] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.701365] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.706586] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.710288] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.716159] iommu: Default domain type: Passthrough [ 0.718886] SCSI subsystem initialized [ 0.721242] ACPI: bus type USB registered [ 0.723132] usbcore: registered new interface driver usbfs [ 0.725110] usbcore: registered new interface driver hub [ 0.727488] usbcore: registered new device driver usb [ 0.730246] pps_core: LinuxPPS API ver. 1 registered [ 0.732016] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.736048] PTP clock support registered [ 0.739185] EDAC MC: Ver: 3.0.0 [ 0.742126] PCI: Using ACPI for IRQ routing [ 0.744996] NetLabel: Initializing [ 0.747017] NetLabel: domain hash size = 128 [ 0.751014] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.753074] NetLabel: unlabeled traffic allowed by default [ 0.758039] vgaarb: loaded [ 0.759276] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.761015] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.775978] clocksource: Switched to clocksource kvm-clock [ 0.939728] VFS: Disk quotas dquot_6.6.0 [ 0.942269] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.945890] *** VALIDATE ramfs *** [ 0.947800] *** VALIDATE hugetlbfs *** [ 0.950306] pnp: PnP ACPI init [ 0.954338] pnp: PnP ACPI: found 6 devices [ 0.977162] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.980921] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.985073] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.988683] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.993118] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.996088] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.999659] NET: Registered protocol family 2 [ 1.002374] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.010352] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.015976] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.024533] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.029595] TCP: Hash tables configured (established 65536 bind 65536) [ 1.034161] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.039479] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.043266] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.047877] NET: Registered protocol family 1 [ 1.050744] RPC: Registered named UNIX socket transport module. [ 1.053862] RPC: Registered udp transport module. [ 1.057384] RPC: Registered tcp transport module. [ 1.059487] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.062517] NET: Registered protocol family 44 [ 1.064712] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.068314] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.071887] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.075821] PCI: CLS 0 bytes, default 64 [ 1.079868] Unpacking initramfs... [ 2.728117] debug: unmapping init [mem 0xffff91ba3cc54000-0xffff91ba3ffbffff] [ 2.732906] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.738621] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.741485] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.278208] Initialise system trusted keyrings [ 3.281256] Key type blacklist registered [ 3.283217] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.293036] zbud: loaded [ 3.296185] *** VALIDATE nfs *** [ 3.297683] *** VALIDATE nfs4 *** [ 3.299418] pstore: using deflate compression [ 3.303517] Platform Keyring initialized [ 3.441448] NET: Registered protocol family 38 [ 3.444624] Key type asymmetric registered [ 3.449564] Asymmetric key parser 'x509' registered [ 3.452238] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.456282] io scheduler mq-deadline registered [ 3.458605] io scheduler kyber registered [ 3.460918] io scheduler bfq registered [ 3.463503] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.467493] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.471231] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.474597] ACPI: Power Button [PWRF] [ 3.480582] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.489257] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.504863] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.519571] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.584195] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.622482] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.658025] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.667884] Non-volatile memory driver v1.3 [ 3.672161] Linux agpgart interface v0.103 [ 3.719915] virtio_blk virtio1: [vda] 146168 512-byte logical blocks (74.8 MB/71.4 MiB) [ 3.725816] vda: detected capacity change from 0 to 74838016 [ 3.761850] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.766336] vdb: detected capacity change from 0 to 1073741824 [ 3.786470] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.791932] vdc: detected capacity change from 0 to 2621440000 [ 3.812816] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.820287] vdd: detected capacity change from 0 to 2621440000 [ 3.838761] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.842545] vde: detected capacity change from 0 to 4294967296 [ 3.863585] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.867647] vdf: detected capacity change from 0 to 4294967296 [ 3.876957] libphy: Fixed MDIO Bus: probed [ 3.884565] usbcore: registered new interface driver usbserial_generic [ 3.889326] usbserial: USB Serial support registered for generic [ 3.893646] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.900779] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.904381] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.909616] mousedev: PS/2 mouse device common for all mice [ 3.912622] rtc_cmos 00:05: RTC can wake from S4 [ 3.917336] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.922974] rtc_cmos 00:05: registered as rtc0 [ 3.926857] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.927780] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.934847] intel_pstate: CPU model not supported [ 3.938248] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.942768] hid: raw HID events driver (C) Jiri Kosina [ 3.945234] usbcore: registered new interface driver usbhid [ 3.947504] usbhid: USB HID core driver [ 3.949352] drop_monitor: Initializing network drop monitor service [ 3.952088] Initializing XFRM netlink socket [ 3.955525] NET: Registered protocol family 10 [ 3.958576] Segment Routing with IPv6 [ 3.960579] NET: Registered protocol family 17 [ 3.964174] mpls_gso: MPLS GSO support [ 3.968947] RAS: Correctable Errors collector initialized. [ 3.971055] AVX version of gcm_enc/dec engaged. [ 3.973993] AES CTR mode by8 optimization enabled [ 4.075466] sched_clock: Marking stable (4075445693, 0)->(5172110070, -1096664377) [ 4.080191] registered taskstats version 1 [ 4.082293] Loading compiled-in X.509 certificates [ 4.084624] zswap: loaded using pool lzo/zbud [ 4.112137] Key type big_key registered [ 4.128917] Key type encrypted registered [ 4.130390] ima: No TPM chip found, activating TPM-bypass! [ 4.132608] ima: Allocated hash algorithm: sha1 [ 4.134433] ima: No architecture policies found [ 4.136625] evm: Initialising EVM extended attributes: [ 4.138764] evm: security.selinux [ 4.140280] evm: security.ima [ 4.141603] evm: security.capability [ 4.143163] evm: HMAC attrs: 0x1 [ 4.146401] rtc_cmos 00:05: setting system clock to 2026-08-24 08:31:09 UTC (1787560269) [ 4.153597] debug: unmapping init [mem 0xffffffff9a203000-0xffffffff9a3fffff] [ 4.157282] debug: unmapping init [mem 0xffffffff98f82000-0xffffffff99258fff] [ 4.163075] Write protecting the kernel read-only data: 28672k [ 4.167241] debug: unmapping init [mem 0xffffffff97603000-0xffffffff977fffff] [ 4.170302] debug: unmapping init [mem 0xffffffff97f14000-0xffffffff97ffffff] [ 4.207072] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 4.216047] systemd[1]: Detected virtualization kvm. [ 4.217957] systemd[1]: Detected architecture x86-64. [ 4.219740] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 4.244383] systemd[1]: No hostname configured. [ 4.246962] systemd[1]: Set hostname to . [ 4.249735] random: systemd: uninitialized urandom read (16 bytes read) [ 4.253274] systemd[1]: Initializing machine ID from random generator. [ 4.328834] random: ln: uninitialized urandom read (6 bytes read) [ 4.460664] random: systemd: uninitialized urandom read (16 bytes read) [ 4.463506] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ 4.468624] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 4.473291] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ OK ] Reached target Swap. Starting Setup Virtual Console... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Initrd Root Device. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... Starting Create Volatile Files and Directories... [ OK ] Reached target Slices. [ OK ] Reached target Timers. Starting Apply Kernel Variables... [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Started Memstrack Anylazing Service. [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 5.151887] device-mapper: uevent: version 1.0.3 [ 5.154240] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.943516] virtio_net virtio0 ens2: renamed from eth0 [ 5.955176] random: fast init done [ 6.099729] scsi host0: ata_piix [ 6.107378] scsi host1: ata_piix [ 6.112399] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 6.115138] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 10.650278] random: crng init done [ 10.651534] random: 7 urandom warning(s) missed due to ratelimiting [ 10.902356] dracut-initqueue[593]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 12.029748] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Slices. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 15.149714] printk: systemd: 26 output lines suppressed due to ratelimiting [ 16.071239] SELinux: Disabled at runtime. [ 16.196128] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 16.211069] systemd[1]: Detected virtualization kvm. [ 16.213623] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 17.758541] systemd[1]: initrd-switch-root.service: Succeeded. [ 17.763542] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 17.774924] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 17.785322] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 17.792587] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 17.803167] systemd[1]: Starting Journal Service... Starting Journal Service... [ 17.822378] systemd[1]: Created slice system-serial\x2dgetty.slice. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on Process Core Dump Socket. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on udev Control Socket. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Created slice system-getty.slice. Mounting POSIX Message Queue File System... [ OK ] Created slice User and Session Slice. Starting Remount Root and Kernel File Systems... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. Mounting Kernel Debug File System... Mounting Huge Pages File System... [ OK ] Reached target rpc_pipefs.target. Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [ 18.294737] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Starting Apply Kernel Variables... [ OK ] Reached target Slices. [ OK ] Started Journal Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted POSIX Message Queue File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted Huge Pages File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Apply Kernel Variables. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 19.319079] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 20.549299] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 20.761852] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 21.309694] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 21.404988] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (7s / no limit) [** ] A start job is running for Configur…-only root support (8s / no limit) [*** ] A start job is running for Configur…-only root support (8s / no limit) [ *** ] A start job is running for Configur…-only root support (9s / no limit)[ 27.319950] Key type dns_resolver registered [ *** ] A start job is running for Configur…-only root support (9s / no limit) [ ***] A start job is running for Configur…only root support (10s / no limit) [ **] A start job is running for Configur…only root support (10s / no limit)[ 28.516575] NFS: Registering the id_resolver key type [ 28.531940] Key type id_resolver registered [ 28.541080] Key type id_legacy registered [ *] A start job is running for Configur…only root support (11s / no limit) [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... Starting Login Service... [ OK ] Reached target sshd-keygen.target. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting Hostname Service... [ OK ] Started Login Service. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg659-server login: [ 44.189772] hrtimer: interrupt took 4685839 ns [ 85.109859] libcfs: loading out-of-tree module taints kernel. [ 85.188901] Key type ._llcrypt registered [ 85.192648] Key type .llcrypt registered [ 85.338358] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_hostid [ 105.687443] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 107.108729] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 107.121822] alg: No test for adler32 (adler32-zlib) [ 108.825085] Lustre: Lustre: Build Version: 2.17.57_82_g4029ffa [ 109.865992] LNet: Added LNI 192.168.206.159@tcp [8/256/0/180] [ 111.624202] Key type lgssc registered [ 113.616398] Lustre: Echo OBD driver; http://www.lustre.org/ [ 132.308958] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 171.690556] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 183.789505] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 183.821423] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 185.088712] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 185.133513] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 185.220758] Lustre: lustre-MDT0000: new disk, initializing [ 185.343784] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 185.354286] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 189.514723] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 201.608600] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 201.756467] Lustre: 6511:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 201.801562] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 201.810645] Lustre: Skipped 1 previous similar message [ 201.943937] Lustre: lustre-MDT0001: new disk, initializing [ 201.996141] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 202.022302] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 202.032350] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 206.191261] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 210.460324] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 218.183245] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 218.368592] Lustre: lustre-OST0000: new disk, initializing [ 218.372620] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 218.380122] Lustre: 8450:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 218.432385] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 222.420174] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 222.430952] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 222.514174] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 223.562599] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 237.224246] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 237.325993] Lustre: lustre-OST0001: new disk, initializing [ 237.329417] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 237.334031] Lustre: 9521:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 237.387243] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 243.237184] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 244.328827] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 244.340592] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 244.446066] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 255.411809] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 261.914130] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 267.656102] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing check_logdir /tmp/testlogs/ [ 271.586541] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing yml_node [ 275.118663] Lustre: DEBUG MARKER: Client: 2.17.57.82 [ 277.105447] Lustre: DEBUG MARKER: MDS: 2.17.57.82 [ 279.137908] Lustre: DEBUG MARKER: OSS: 2.17.57.82 [ 280.765586] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Mon Aug 24 04:35:45 EDT 2026 [ 295.646685] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 297.182516] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 314.851152] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 314.855554] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 314.867323] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 316.399735] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 316.399749] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 316.409993] Lustre: Skipped 1 previous similar message [ 316.434415] Lustre: Skipped 1 previous similar message [ 318.683744] Lustre: server umount lustre-MDT0000 complete [ 321.952158] LustreError: 6503:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787560587 with bad export cookie 11594590559546313467 [ 321.952886] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 321.971862] LustreError: 6503:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 326.624714] LustreError: 6518:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 326.625143] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 326.635763] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 326.635771] Lustre: Skipped 2 previous similar messages [ 326.645252] LustreError: 6518:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 326.657843] Lustre: Skipped 2 previous similar messages [ 331.746891] LustreError: 6520:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 331.752813] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 331.791748] Lustre: Skipped 2 previous similar messages [ 332.770624] LustreError: 6519:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 332.793045] LustreError: 6519:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 336.353072] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 336.663881] Lustre: server umount lustre-MDT0001 complete [ 341.267347] Lustre: server umount lustre-OST0000 complete [ 346.833078] Lustre: server umount lustre-OST0001 complete [ 353.430082] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_hostid [ 363.714099] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 397.376787] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 397.601522] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 397.634394] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 397.758763] Lustre: lustre-MDT0000: new disk, initializing [ 397.871648] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 397.917951] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 401.991206] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 413.175422] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 413.493213] Lustre: 15072:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 413.583402] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 413.585877] Lustre: Skipped 1 previous similar message [ 413.643614] Lustre: lustre-MDT0001: new disk, initializing [ 413.744203] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 413.776624] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 413.787070] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 418.018727] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 428.839145] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 435.834223] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 443.283550] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 443.579400] Lustre: lustre-OST0000: new disk, initializing [ 443.582639] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 443.589265] Lustre: 17040:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 443.677748] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 445.661627] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 445.686831] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 445.794621] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 450.832093] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 459.653467] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 463.949224] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 464.163468] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 467.963381] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 468.137865] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 471.528205] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 471.535504] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 471.544799] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 471.544806] Lustre: Skipped 2 previous similar messages [ 471.563096] LustreError: Skipped 1 previous similar message [ 475.948403] Lustre: server umount lustre-OST0000 complete [ 480.225671] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 480.231663] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 480.241537] Lustre: Skipped 1 previous similar message [ 480.252530] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 480.259148] Lustre: Skipped 1 previous similar message [ 485.186744] Lustre: server umount lustre-MDT0000 complete [ 486.887665] LustreError: 15081:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 486.908880] LustreError: 15081:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 488.524619] LustreError: 17050:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787560753 with bad export cookie 11594590559546315000 [ 488.527902] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 488.532189] LustreError: 17050:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 488.794827] Lustre: server umount lustre-MDT0001 complete [ 492.719715] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 04:39:17 (1787560757) [ 498.334452] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 498.777557] LustreError: 19308:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 498.838983] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 502.970105] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 509.413672] LustreError: 19309:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 509.447283] LustreError: 19309:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 510.668558] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 515.085756] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 522.596158] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 529.492609] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 536.701181] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 542.479617] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 550.523575] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 562.715607] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 571.876062] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 571.881646] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 571.887723] Lustre: Skipped 2 previous similar messages [ 571.895431] Lustre: Skipped 2 previous similar messages [ 576.582735] Lustre: server umount lustre-MDT0000 complete [ 576.995719] LustreError: 19309:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 577.015514] LustreError: 19309:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 580.309144] LustreError: 19288:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787560845 with bad export cookie 11594590559546315882 [ 580.320625] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 580.323160] LustreError: 19288:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 580.642597] Lustre: server umount lustre-MDT0001 complete [ 594.899367] Lustre: server umount lustre-OST0000 complete [ 617.542737] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 627.978108] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 628.586638] LustreError: 23713:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 628.641841] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 628.654163] Lustre: Skipped 2 previous similar messages [ 632.870492] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 640.846252] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 645.725565] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 648.332428] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 654.183274] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 660.207660] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 668.153316] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 668.330473] Lustre: lustre-OST0001: new disk, initializing [ 668.337423] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 668.348408] Lustre: 26026:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 669.832882] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 669.854428] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 669.973150] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 675.303589] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 684.824946] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 689.670730] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 707.553276] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 707.566855] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 707.593659] Lustre: Skipped 1 previous similar message [ 707.611631] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 707.626635] Lustre: Skipped 1 previous similar message [ 711.445934] Lustre: server umount lustre-MDT0000 complete [ 714.213147] LustreError: 23709:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 714.235890] LustreError: 23709:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 715.420568] LustreError: 23695:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787560980 with bad export cookie 11594590559546316729 [ 715.425541] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 715.440693] LustreError: 23695:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 715.964984] Lustre: server umount lustre-MDT0001 complete [ 730.567016] Lustre: server umount lustre-OST0000 complete [ 744.400946] Lustre: server umount lustre-OST0001 complete [ 760.550416] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 770.394431] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 770.975808] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 770.982433] Lustre: Skipped 3 previous similar messages [ 775.220563] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 785.032463] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 789.714379] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 792.122848] Lustre: 30198:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 799.302386] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 806.145876] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 815.254179] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 821.047078] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 829.520837] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 833.289683] Lustre: 32075:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 841.700832] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 841.712129] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 841.728495] Lustre: Skipped 3 previous similar messages [ 841.736339] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 841.743127] Lustre: Skipped 4 previous similar messages [ 846.271235] Lustre: server umount lustre-MDT0000 complete [ 849.390580] LustreError: 31898:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 849.409353] LustreError: 31898:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [ 849.884492] LustreError: 29030:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787561115 with bad export cookie 11594590559546318017 [ 849.886595] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 849.898374] LustreError: 29030:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 850.190796] Lustre: server umount lustre-MDT0001 complete [ 863.841519] Lustre: server umount lustre-OST0000 complete [ 877.590713] Lustre: server umount lustre-OST0001 complete [ 885.542982] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 04:45:50 (1787561150) [ 909.877996] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 918.690175] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 922.802048] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 930.956279] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 935.403636] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 938.199602] Lustre: 36026:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 944.999697] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 951.164393] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 958.820741] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 965.021744] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 972.889151] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 976.608969] Lustre: 37905:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 982.501129] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 982.511258] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 982.529232] Lustre: Skipped 3 previous similar messages [ 982.538593] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 982.546588] Lustre: Skipped 4 previous similar messages [ 987.610042] Lustre: server umount lustre-MDT0000 complete [ 991.203162] LustreError: 34860:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787561256 with bad export cookie 11594590559546319130 [ 991.217065] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 991.218903] LustreError: 34860:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 991.688478] Lustre: server umount lustre-MDT0001 complete [ 1004.882659] Lustre: server umount lustre-OST0000 complete [ 1018.754132] Lustre: server umount lustre-OST0001 complete [ 1028.263666] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 04:48:12 (1787561292) [ 1045.247726] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 1056.748244] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1057.548079] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1057.561919] Lustre: Skipped 7 previous similar messages [ 1062.696455] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 1072.368741] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1079.313138] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 1082.517792] Lustre: 41199:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1089.427366] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1097.108956] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 1105.272983] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1111.686559] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 1118.874164] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1122.621316] Lustre: 43077:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1123.649899] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1123.661152] Lustre: Skipped 1 previous similar message [ 1129.441545] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1129.462283] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1129.483647] Lustre: Skipped 3 previous similar messages [ 1133.300226] Lustre: server umount lustre-MDT0000 complete [ 1134.618226] LustreError: 40046:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.206.59@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1134.652899] LustreError: 40046:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 15 previous similar messages [ 1137.197292] LustreError: 40030:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787561402 with bad export cookie 11594590559546320236 [ 1137.203677] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1137.211392] LustreError: 40030:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1143.599429] Lustre: server umount lustre-MDT0001 complete [ 1151.971296] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1161.699202] LustreError: 3642:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff91baadb28a80 x1874392908165504/t0(0) o250->MGC192.168.206.159@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1166.503712] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 1172.396200] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1174.365524] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1178.470725] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 1180.150499] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1180.161787] Lustre: Skipped 1 previous similar message [ 1184.936880] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1186.451418] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1191.873442] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1193.265874] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1200.098399] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1205.403214] Lustre: server umount lustre-MDT0000 complete [ 1209.349891] LustreError: 40030:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787561474 with bad export cookie 11594590559546321363 [ 1209.362907] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1209.371777] LustreError: 40030:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1209.817798] Lustre: server umount lustre-MDT0001 complete [ 1224.384757] Lustre: server umount lustre-OST0000 complete [ 1227.232263] Lustre: 3646:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787561476/real 1787561476] req@ffff91baacdcad80 x1874392908199040/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787561492 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1228.722845] Lustre: server umount lustre-OST0001 complete [ 1238.782919] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 04:51:43 (1787561503) [ 1257.367626] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 1269.491413] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1276.479219] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 1284.611385] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1289.832652] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 1292.737592] Lustre: 48832:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1299.572984] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1307.440577] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 1318.286982] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1325.296764] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 1333.322527] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1338.485340] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1351.650574] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1351.662726] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1351.666151] Lustre: Skipped 19 previous similar messages [ 1352.193990] Lustre: server umount lustre-MDT0000 complete [ 1356.386591] LustreError: 47662:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787561621 with bad export cookie 11594590559546322154 [ 1356.391399] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1356.405902] LustreError: 47662:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1356.877197] Lustre: server umount lustre-MDT0001 complete [ 1371.070791] Lustre: server umount lustre-OST0000 complete [ 1386.443837] Lustre: server umount lustre-OST0001 complete [ 1397.744649] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 04:54:21 (1787561661) [ 1401.039671] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1402.772555] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 04:54:27 (1787561667) [ 1414.612328] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_hostid [ 1424.683806] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 1448.265758] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1448.513237] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1448.569338] Lustre: lustre-MDT0000: new disk, initializing [ 1448.629190] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1452.862248] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 1462.728575] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1469.346136] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1469.723291] Lustre: lustre-OST0000: new disk, initializing [ 1469.726640] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1469.731826] Lustre: Skipped 1 previous similar message [ 1469.741737] Lustre: 54910:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1471.773315] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1471.782339] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1471.828272] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1476.604647] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 1485.817585] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1488.689915] Lustre: DEBUG MARKER: create test files [ 1523.171956] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1523.193182] Lustre: Skipped 14 previous similar messages [ 1529.348210] Lustre: server umount lustre-OST0000 complete [ 1533.486857] Lustre: server umount lustre-MDT0000 complete [ 1544.091480] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing unload_modules_local [ 1546.847638] Key type lgssc unregistered [ 1547.110842] LNet: 56548:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1547.118739] LNetError: 56548:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1547.142487] LNet: Removed LNI 192.168.206.159@tcp [ 1548.013155] Key type .llcrypt unregistered [ 1548.016197] Key type ._llcrypt unregistered [ 1581.122500] Key type ._llcrypt registered [ 1581.124703] Key type .llcrypt registered [ 1581.284133] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 1582.689044] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1582.734521] alg: No test for adler32 (adler32-zlib) [ 1583.857482] Lustre: Lustre: Build Version: 2.17.57_82_g4029ffa [ 1584.137929] LNet: Added LNI 192.168.206.159@tcp [8/256/0/180] [ 1585.865165] Key type lgssc registered [ 1587.040729] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1594.575525] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1594.601200] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1596.241995] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1600.911911] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 1609.312572] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1616.326320] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1616.627393] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1620.729541] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:101 to 0x240000400:129) [ 1620.859775] Lustre: 59283:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1620.865967] Lustre: 59283:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1620.869473] Lustre: 59283:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1620.873604] Lustre: 59283:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1620.877325] Lustre: 59283:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1620.880854] Lustre: 59283:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1623.315796] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 1633.256475] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1640.167563] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1645.337732] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1656.820132] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1656.826244] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1656.856789] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1659.550224] Lustre: server umount lustre-OST0000 complete [ 1664.152242] Lustre: server umount lustre-MDT0000 complete [ 1676.999217] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing unload_modules_local [ 1680.037687] Key type lgssc unregistered [ 1680.404845] LNet: 60894:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1680.412480] LNetError: 60894:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1680.424976] LNet: Removed LNI 192.168.206.159@tcp [ 1681.446580] Key type .llcrypt unregistered [ 1681.449935] Key type ._llcrypt unregistered [ 1712.509956] Key type ._llcrypt registered [ 1712.513124] Key type .llcrypt registered [ 1712.640787] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 1713.809560] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1713.891870] alg: No test for adler32 (adler32-zlib) [ 1714.853784] Lustre: Lustre: Build Version: 2.17.57_82_g4029ffa [ 1715.138768] LNet: Added LNI 192.168.206.159@tcp [8/256/0/180] [ 1716.816163] Key type lgssc registered [ 1717.808533] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1725.172066] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1725.194706] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1726.739538] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1731.758320] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 1741.400228] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1748.747641] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1749.226472] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1752.315466] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1752.373441] Lustre: 63729:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1752.383726] Lustre: 63729:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1752.391047] Lustre: 63729:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1752.397796] Lustre: 63729:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1752.405538] Lustre: 63729:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1752.411663] Lustre: 63729:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1755.653447] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 1763.034090] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1767.649807] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1808.868203] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1808.878847] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1808.898481] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1810.868987] Lustre: server umount lustre-OST0000 complete [ 1815.531397] Lustre: server umount lustre-MDT0000 complete [ 1826.173864] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing unload_modules_local [ 1828.763892] Key type lgssc unregistered [ 1829.119823] LNet: 65327:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1829.131449] LNetError: 65327:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1829.159395] LNet: Removed LNI 192.168.206.159@tcp [ 1830.263585] Key type .llcrypt unregistered [ 1830.271497] Key type ._llcrypt unregistered [ 1846.831432] Key type ._llcrypt registered [ 1846.833859] Key type .llcrypt registered [ 1846.915753] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_hostid [ 1861.300299] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 1862.507305] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1862.532646] alg: No test for adler32 (adler32-zlib) [ 1863.578312] Lustre: Lustre: Build Version: 2.17.57_82_g4029ffa [ 1863.810107] LNet: Added LNI 192.168.206.159@tcp [8/256/0/180] [ 1865.448161] Key type lgssc registered [ 1866.436648] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1901.473677] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1901.497652] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1902.898985] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1902.955655] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1903.115239] Lustre: lustre-MDT0000: new disk, initializing [ 1903.187721] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1903.203540] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1908.379333] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 1919.920634] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1920.155089] Lustre: 68481:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1920.261965] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1920.271294] Lustre: Skipped 1 previous similar message [ 1920.356127] Lustre: lustre-MDT0001: new disk, initializing [ 1920.526277] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1920.615593] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1920.639941] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1925.813846] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 1935.035859] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1940.860995] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1947.643507] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1947.878896] Lustre: lustre-OST0000: new disk, initializing [ 1947.883230] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1947.889105] Lustre: 70449:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1947.969358] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1953.736426] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 1957.450357] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1957.466990] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1957.640544] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1963.647173] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1968.618328] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1968.817375] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1972.984779] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1973.177831] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1978.338596] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1978.344246] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1978.362300] Lustre: Skipped 1 previous similar message [ 1981.406899] Lustre: server umount lustre-OST0000 complete [ 1987.042067] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1987.051495] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1987.072235] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1987.079264] Lustre: Skipped 1 previous similar message [ 1988.582677] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1988.590854] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1992.942612] Lustre: server umount lustre-MDT0000 complete [ 1997.201700] LustreError: 68471:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787562262 with bad export cookie 13223726333125201014 [ 1997.202417] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1997.215152] LustreError: 68471:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1997.824681] Lustre: server umount lustre-MDT0001 complete [ 2006.694336] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 05:04:31 (1787562271) [ 2018.867071] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 2058.609693] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_hostid [ 2067.909681] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 2102.751987] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2103.015789] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2103.046752] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2103.130278] Lustre: lustre-MDT0000: new disk, initializing [ 2103.196360] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2103.237395] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2107.344930] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 2118.166584] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2118.241457] Lustre: 76468:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2118.320539] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2118.326297] Lustre: Skipped 1 previous similar message [ 2118.490213] Lustre: lustre-MDT0001: new disk, initializing [ 2118.591529] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2118.645260] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2118.661466] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2123.721117] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 2132.858983] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2138.404822] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2145.213100] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2145.418801] Lustre: lustre-OST0000: new disk, initializing [ 2145.425680] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2145.436585] Lustre: 78436:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2145.529278] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2152.173364] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 2153.065134] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2153.084816] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2153.298249] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2163.594271] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2169.639106] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2169.924054] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2175.402605] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2175.681968] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2179.043248] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2179.046368] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2179.048170] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2179.048176] Lustre: Skipped 1 previous similar message [ 2179.072389] LustreError: Skipped 1 previous similar message [ 2179.142688] Lustre: Skipped 1 previous similar message [ 2184.167641] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2184.177702] Lustre: Skipped 1 previous similar message [ 2192.352153] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2192.538093] Lustre: server umount lustre-OST0000 complete [ 2199.533853] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2199.537711] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2199.550815] Lustre: Skipped 1 previous similar message [ 2199.565454] Lustre: Skipped 4 previous similar messages [ 2202.669419] Lustre: server umount lustre-MDT0000 complete [ 2204.642549] LustreError: 76479:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2204.669481] LustreError: 76479:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 2206.094337] LustreError: 76460:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787562471 with bad export cookie 13223726333125201896 [ 2206.106511] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2206.626777] Lustre: server umount lustre-MDT0001 complete [ 2215.271916] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 05:07:59 (1787562479) [ 2222.373444] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2222.954977] LustreError: 80852:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2223.083825] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2228.202825] LustreError: 80853:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2228.237784] LustreError: 80853:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2229.102883] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 2233.317155] LustreError: 80852:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2238.437805] LustreError: 80853:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2238.802489] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2244.068840] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 2252.153810] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2258.411961] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2265.147376] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2265.527778] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2265.533642] Lustre: Skipped 1 previous similar message [ 2271.866743] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 2280.609711] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2285.386824] Lustre: 83525:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting ee4bbb76-b8d7-420a-aad5-11f23622f64a at adminstrative request [ 2289.131843] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2289.153246] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2299.370727] Lustre: MGS: Client ee4bbb76-b8d7-420a-aad5-11f23622f64a (at 0@lo) reconnecting [ 2299.383365] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xb7841d09a4e8135a to 0xb7841d09a4e815fa [ 2299.417817] Lustre: MGC192.168.206.159@tcp: Connection restored to 0@lo (at 0@lo) [ 2322.083940] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2322.234582] Lustre: lustre-OST0001: new disk, initializing [ 2322.240172] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2322.247861] Lustre: 83923:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2328.876577] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 2329.224828] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2329.245546] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2329.408253] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2338.730567] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2344.929513] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2344.933564] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2344.946651] LustreError: Skipped 1 previous similar message [ 2344.947426] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2344.957775] Lustre: Skipped 1 previous similar message [ 2356.192123] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2356.335919] Lustre: server umount lustre-OST0001 complete [ 2360.316884] LustreError: 82672:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2360.335432] LustreError: 82672:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2374.112247] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2374.491200] Lustre: server umount lustre-OST0000 complete [ 2380.770823] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2380.784861] Lustre: Skipped 2 previous similar messages [ 2380.789156] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2380.796359] Lustre: Skipped 12 previous similar messages [ 2384.897471] Lustre: server umount lustre-MDT0000 complete [ 2385.889418] LustreError: 80853:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2385.910459] LustreError: 80853:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 2388.542300] LustreError: 81588:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787562653 with bad export cookie 13223726333125203450 [ 2388.547713] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2388.564686] LustreError: 81588:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 2388.895701] Lustre: server umount lustre-MDT0001 complete [ 2398.672784] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 05:11:02 (1787562662) [ 2400.461150] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2402.345637] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 05:11:06 (1787562666) [ 2404.930942] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2407.497111] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 05:11:11 (1787562671) [ 2409.343433] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2411.665714] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 05:11:15 (1787562675) [ 2413.417837] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2415.906127] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 05:11:19 (1787562679) [ 2418.258372] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2425.192696] loop: module loaded [ 2425.225178] loop0: detected capacity change from 0 to 4294967296 [ 2425.283965] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2426.951299] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2430.867140] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2434.804701] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2462.496612] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2462.552220] LustreError: 88050:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2462.556343] LustreError: 88050:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2462.561246] LustreError: 88050:0:(tgt_mount.c:2582:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2462.567077] LustreError: 88050:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2482.155677] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_hostid [ 2492.628660] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 2530.332378] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2530.493840] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2530.512597] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2530.575731] Lustre: lustre-MDT0000: new disk, initializing [ 2530.652095] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2530.661315] Lustre: Skipped 1 previous similar message [ 2530.675249] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2534.358874] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 2545.074636] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2545.261639] Lustre: 90916:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2545.507059] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2549.236971] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 2558.355574] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2565.058556] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2571.567456] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2571.848351] Lustre: 92881:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2578.845973] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 2580.544815] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2580.680399] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2588.418342] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2592.760421] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2592.881742] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2597.739860] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2598.010589] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2601.445416] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2601.446382] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2601.455241] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2601.457199] LustreError: Skipped 1 previous similar message [ 2601.470354] Lustre: Skipped 2 previous similar messages [ 2606.242950] Lustre: server umount lustre-OST0000 complete [ 2616.438837] Lustre: server umount lustre-MDT0000 complete [ 2616.802302] LustreError: 90926:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2616.823571] LustreError: 90926:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2619.942207] LustreError: 90908:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787562885 with bad export cookie 13223726333125204066 [ 2619.943108] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2619.952390] LustreError: 90908:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2620.215924] Lustre: server umount lustre-MDT0001 complete [ 2628.195892] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 05:14:52 (1787562892) [ 2635.598741] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2640.713548] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 2649.680106] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2654.780133] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 2662.207409] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2667.539355] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2673.972796] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2674.340478] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2674.349099] Lustre: Skipped 4 previous similar messages [ 2679.462414] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 2686.772800] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2691.689510] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2691.943812] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2697.052669] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2697.247056] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2703.957476] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2704.181796] Lustre: lustre-OST0001: new disk, initializing [ 2704.184796] Lustre: Skipped 2 previous similar messages [ 2704.190340] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2704.195631] Lustre: Skipped 3 previous similar messages [ 2704.200643] Lustre: 98664:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2709.536180] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 2711.635211] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2711.643506] Lustre: Skipped 2 previous similar messages [ 2711.646872] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2711.720690] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2718.848663] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2725.388399] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2725.645995] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2731.289485] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2731.636502] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2751.468860] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2772.588311] Lustre: Failing over lustre-MDT0000 [ 2772.962503] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2772.975963] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2772.989030] Lustre: Skipped 2 previous similar messages [ 2772.995266] LustreError: 95297:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2773.021404] LustreError: 95297:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2773.065975] Lustre: server umount lustre-MDT0000 complete [ 2788.817989] Lustre: 66704:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787563038/real 1787563038] req@00000000362f272f x1874394747254272/t0(0) o400->MGC192.168.206.159@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787563054 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2788.880152] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2798.682990] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2798.688528] LDISKFS-fs (dm-0): recovery complete [ 2798.696513] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2799.075389] LustreError: 66702:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@000000009e21d253 x1874394747260544/t0(0) o250->MGC192.168.206.159@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2799.410522] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2801.187263] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2801.191397] Lustre: 98561:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2804.353714] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 2804.718626] Lustre: 96024:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2804.731281] Lustre: 96024:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 2804.742791] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2804.823282] LustreError: 101512:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2805.136119] LustreError: 101512:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2805.481410] LustreError: 101512:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2805.489614] LustreError: 101512:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2805.825073] LustreError: 101512:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2805.837393] LustreError: 101512:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2805.853181] Lustre: 101512:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2805.872980] Lustre: 101512:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 2806.531550] LustreError: 101512:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2806.541189] LustreError: 101512:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2806.864836] LustreError: 101512:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2806.879705] LustreError: 101512:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2808.168144] Lustre: 101512:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2808.195398] Lustre: 101512:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 6 previous similar messages [ 2808.843949] LustreError: 101512:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2808.863625] LustreError: 101512:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2809.192120] LustreError: 101512:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2809.203586] LustreError: 101512:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2812.385054] Lustre: 101512:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2812.400819] Lustre: 101512:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 2813.050992] LustreError: 101512:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2813.068221] LustreError: 101512:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2813.392240] LustreError: 101512:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2813.405099] LustreError: 101512:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2820.398360] Lustre: 101512:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2820.416258] Lustre: 101512:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 24 previous similar messages [ 2821.068226] LustreError: 101512:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2821.087347] LustreError: 101512:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2821.409900] LustreError: 101512:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2821.415432] LustreError: 101512:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2836.644264] Lustre: 101512:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2836.669628] Lustre: 101512:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 50 previous similar messages [ 2837.317195] LustreError: 101512:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2837.327700] LustreError: 101512:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2837.656121] LustreError: 101512:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2837.665400] LustreError: 101512:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2861.104173] Lustre: 101512:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2861.111909] Lustre: 101512:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2861.118624] Lustre: 101512:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client e6d2fd0a-8cbc-4c9c-aa4e-6911f691412c@192.168.206.59@tcp [ 2861.124648] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2861.134170] Lustre: 101512:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@000000009185c991 x1874394732367744/t0(8589934773) o101->e6d2fd0a-8cbc-4c9c-aa4e-6911f691412c@192.168.206.59@tcp:222/0 lens 592/0 e 0 to 0 dl 1787563137 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2861.157603] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2861.166884] Lustre: lustre-MDT0000: Denying connection for new client e6d2fd0a-8cbc-4c9c-aa4e-6911f691412c (at 192.168.206.59@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2861.179860] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2861.219218] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2861.228501] Lustre: Skipped 2 previous similar messages [ 2861.233747] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2861.297787] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:78 to 0x280000401:577) [ 2861.313507] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:545) [ 2876.399922] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2876.404669] Lustre: Skipped 3 previous similar messages [ 2882.285771] Lustre: server umount lustre-OST0000 complete [ 2886.113400] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2892.184193] Lustre: server umount lustre-OST0001 complete [ 2902.696740] Lustre: server umount lustre-MDT0000 complete [ 2906.932891] LustreError: 95277:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787563172 with bad export cookie 13223726333125255278 [ 2906.941803] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2906.957997] LustreError: 95277:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2907.296050] Lustre: server umount lustre-MDT0001 complete [ 2915.244730] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 05:19:39 (1787563179) [ 2922.757359] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2922.763425] Lustre: *** cfs_fail_loc=197, val=0*** [ 2938.272627] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2943.393945] LustreError: 103485:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.159@tcp: failed processing log, type 4: rc = -110 [ 2948.539854] Lustre: *** cfs_fail_loc=197, val=0*** [ 2948.543567] Lustre: Skipped 65 previous similar messages [ 2969.121126] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2969.138604] Lustre: Skipped 2 previous similar messages [ 2975.600996] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 2983.549793] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2985.872461] Lustre: server umount lustre-OST0000 complete [ 2994.682969] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 05:20:59 (1787563259) [ 3017.468234] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3018.270616] LustreError: 105108:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3018.296214] LustreError: 105108:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 40 previous similar messages [ 3023.196946] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 3034.766579] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3038.829223] LustreError: 105139:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3038.841081] LustreError: 105139:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3038.847688] LustreError: 105139:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 21, retries 0, failed: rc = -5 [ 3039.282474] Lustre: server umount lustre-MDT0000 complete [ 3069.690831] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_hostid [ 3079.128760] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 3112.535701] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3113.006996] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3113.081136] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3113.324495] Lustre: lustre-MDT0000: new disk, initializing [ 3113.594385] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3118.841634] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 3133.320235] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3133.592662] Lustre: 109313:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3134.130278] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3139.711569] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 3151.258261] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3157.827784] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3164.612669] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3164.803600] Lustre: 111282:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3171.150293] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 3173.617185] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3180.437713] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3184.978379] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3185.154633] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3189.138845] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3189.426811] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3194.340052] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3194.345278] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3194.359180] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3194.379227] Lustre: Skipped 10 previous similar messages [ 3194.401193] Lustre: Skipped 11 previous similar messages [ 3197.399533] Lustre: server umount lustre-OST0000 complete [ 3215.328284] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3215.729174] Lustre: server umount lustre-MDT0000 complete [ 3220.081777] LustreError: 109305:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787563485 with bad export cookie 13223726333125265897 [ 3220.093801] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3220.098489] LustreError: 109305:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3220.417355] Lustre: server umount lustre-MDT0001 complete [ 3229.229812] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 05:24:53 (1787563493) [ 3238.068357] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing unload_modules_local [ 3240.786410] Key type lgssc unregistered [ 3241.021636] LNet: 113771:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3241.034230] LNetError: 113771:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3241.060628] LNet: Removed LNI 192.168.206.159@tcp [ 3241.906433] Key type .llcrypt unregistered [ 3241.911207] Key type ._llcrypt unregistered [ 3259.133556] Key type ._llcrypt registered [ 3259.135907] Key type .llcrypt registered [ 3259.247923] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_hostid [ 3274.812556] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 3275.763544] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3275.791616] alg: No test for adler32 (adler32-zlib) [ 3277.207339] Lustre: Lustre: Build Version: 2.17.57_82_g4029ffa [ 3277.712963] LNet: Added LNI 192.168.206.159@tcp [8/256/0/180] [ 3279.416674] Key type lgssc registered [ 3280.655751] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3321.134876] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3321.188508] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3322.448878] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3322.535652] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3322.671489] Lustre: lustre-MDT0000: new disk, initializing [ 3322.765066] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3322.783450] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3326.652967] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 3336.971720] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3337.228607] Lustre: lustre-OST0000: new disk, initializing [ 3337.232616] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3337.237071] Lustre: Skipped 1 previous similar message [ 3337.241610] Lustre: 117947:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3337.342914] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3343.443680] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3343.451845] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3343.485708] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3343.866934] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 3353.948733] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3354.110300] Lustre: lustre-OST0001: new disk, initializing [ 3354.114751] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3354.122628] Lustre: 118811:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3354.210676] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3360.963266] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 3362.963460] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3362.973652] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3362.997561] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3373.542235] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3373.547209] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3373.574990] Lustre: Skipped 1 previous similar message [ 3376.043984] Lustre: server umount lustre-MDT0000 complete [ 3384.418597] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3389.784136] LustreError: 117132:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787563655 with bad export cookie 14805795135274227047 [ 3389.791782] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3389.799229] LustreError: 117132:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3389.871936] Lustre: server umount lustre-OST0000 complete [ 3394.050178] Lustre: server umount lustre-OST0001 complete [ 3403.302690] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 05:27:47 (1787563667) [ 3429.778290] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 05:28:14 (1787563694) [ 3440.810882] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 3463.134840] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 05:28:47 (1787563727) [ 3500.910762] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_hostid [ 3510.495885] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 3542.920584] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3543.255153] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3543.293953] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3543.425694] Lustre: lustre-MDT0000: new disk, initializing [ 3543.567834] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3543.605823] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3547.745607] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 3558.186250] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3558.314848] Lustre: 126559:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3558.353962] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3558.359020] Lustre: Skipped 1 previous similar message [ 3558.449694] Lustre: lustre-MDT0001: new disk, initializing [ 3558.555916] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3558.602101] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3558.616259] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3563.239363] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 3572.968356] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3578.901762] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3585.681205] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3585.929113] Lustre: lustre-OST0000: new disk, initializing [ 3585.938447] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3585.947115] Lustre: 128531:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3586.024816] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3592.411971] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 3593.306453] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3593.315409] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3593.495029] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3601.624658] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3606.825170] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3607.080225] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3611.520542] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3611.684905] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3614.180070] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3614.180642] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3614.189929] LustreError: Skipped 1 previous similar message [ 3614.190759] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3614.190763] Lustre: Skipped 1 previous similar message [ 3614.246893] Lustre: Skipped 1 previous similar message [ 3619.299209] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3619.305854] Lustre: Skipped 2 previous similar messages [ 3619.623675] Lustre: server umount lustre-OST0000 complete [ 3624.417454] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3624.419816] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3624.431388] Lustre: Skipped 1 previous similar message [ 3624.457297] Lustre: Skipped 1 previous similar message [ 3629.544259] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3629.551609] Lustre: Skipped 1 previous similar message [ 3637.216224] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3637.516599] Lustre: server umount lustre-MDT0000 complete [ 3641.337242] LustreError: 127355:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787563906 with bad export cookie 14805795135274228272 [ 3641.339686] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3641.356433] LustreError: 127355:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3641.891654] Lustre: server umount lustre-MDT0001 complete [ 3650.981907] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 05:31:55 (1787563915) [ 3657.856552] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3658.398535] LustreError: 130943:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3658.420601] LustreError: 130943:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 3658.503517] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3663.008171] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 3663.849972] LustreError: 130944:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3668.015237] LustreError: 130943:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3671.688707] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3677.275610] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 3685.913731] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3692.703992] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3700.754773] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3701.133551] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3701.146173] Lustre: Skipped 1 previous similar message [ 3707.485424] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 3716.147155] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3724.500194] Lustre: *** cfs_fail_loc=159, val=0*** [ 3740.700112] Lustre: lustre-MDT0000: Client b5a4e00a-53e4-4a9b-8899-7bf264f4ed56 (at 192.168.206.59@tcp) reconnecting [ 3742.679517] Lustre: *** cfs_fail_loc=159, val=0*** [ 3742.685667] Lustre: Skipped 5 previous similar messages [ 3758.207925] Lustre: lustre-MDT0000: Client b5a4e00a-53e4-4a9b-8899-7bf264f4ed56 (at 192.168.206.59@tcp) reconnecting [ 3763.694115] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3763.699643] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3763.719696] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3763.744546] Lustre: Skipped 3 previous similar messages [ 3765.640793] Lustre: server umount lustre-OST0000 complete [ 3773.922916] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3773.938236] Lustre: Skipped 1 previous similar message [ 3776.697236] Lustre: server umount lustre-MDT0000 complete [ 3779.042126] LustreError: 130943:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3780.554338] LustreError: 130923:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787564045 with bad export cookie 14805795135274229154 [ 3780.574775] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3780.576194] LustreError: 130923:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3780.992437] Lustre: server umount lustre-MDT0001 complete [ 3794.891255] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing unload_modules_local [ 3798.518813] Key type lgssc unregistered [ 3798.905966] LNet: 134721:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3798.923996] LNetError: 134721:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3798.947502] LNet: Removed LNI 192.168.206.159@tcp [ 3800.295775] Key type .llcrypt unregistered [ 3800.299267] Key type ._llcrypt unregistered [ 3827.682467] Key type ._llcrypt registered [ 3827.684628] Key type .llcrypt registered [ 3827.823118] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 3829.036744] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3829.178220] alg: No test for adler32 (adler32-zlib) [ 3830.347412] Lustre: Lustre: Build Version: 2.17.57_82_g4029ffa [ 3830.614817] LNet: Added LNI 192.168.206.159@tcp [8/256/0/180] [ 3832.312173] Key type lgssc registered [ 3833.838216] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3841.565249] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3841.598765] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3843.099143] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3847.622984] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 3853.286631] LustreError: 136136:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3853.304323] LustreError: 136136:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3856.833291] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3857.259880] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3861.776177] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 3868.564369] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3874.633605] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3881.685848] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3881.898953] LustreError: 137955:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3882.029752] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3887.828299] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 3888.129944] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3894.444334] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3904.357073] Lustre: *** cfs_fail_loc=159, val=0*** [ 3920.981669] Lustre: lustre-MDT0000: Client 1d006c53-e5bf-4103-95e2-de6acf22ff94 (at 192.168.206.59@tcp) reconnecting [ 3923.076317] Lustre: *** cfs_fail_loc=159, val=0*** [ 3923.080227] Lustre: Skipped 1 previous similar message [ 3939.401527] Lustre: lustre-MDT0001: Client 1d006c53-e5bf-4103-95e2-de6acf22ff94 (at 192.168.206.59@tcp) reconnecting [ 3940.797838] Lustre: *** cfs_fail_loc=159, val=0*** [ 3940.802341] Lustre: Skipped 2 previous similar messages [ 3956.256841] Lustre: lustre-MDT0001: Client 1d006c53-e5bf-4103-95e2-de6acf22ff94 (at 192.168.206.59@tcp) reconnecting [ 3972.667765] Lustre: 139199:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3975.616774] Lustre: *** cfs_fail_loc=159, val=0*** [ 3975.621095] Lustre: Skipped 4 previous similar messages [ 3992.164422] Lustre: lustre-MDT0000: Client 39156e9d-0e4e-4f55-8e2f-e535db4d21aa (at 192.168.206.59@tcp) reconnecting [ 3994.393453] Lustre: *** cfs_fail_loc=159, val=0*** [ 3994.403221] Lustre: Skipped 15 previous similar messages [ 4010.591699] Lustre: lustre-MDT0000: Client 39156e9d-0e4e-4f55-8e2f-e535db4d21aa (at 192.168.206.59@tcp) reconnecting [ 4012.092535] Lustre: 139493:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4018.180060] LustreError: lustre-OST0000-osc-MDT0001: operation ost_destroy to node 0@lo failed: rc = -107 [ 4018.183748] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4018.199679] LustreError: Skipped 1 previous similar message [ 4018.231240] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4021.220236] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4021.222577] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4021.234220] LustreError: Skipped 4 previous similar messages [ 4021.234262] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4021.264313] Lustre: Skipped 1 previous similar message [ 4024.317919] Lustre: server umount lustre-OST0000 complete [ 4031.049772] Lustre: server umount lustre-MDT0000 complete [ 4031.457849] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4031.460534] LustreError: 138891:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4031.472284] Lustre: Skipped 1 previous similar message [ 4031.496526] LustreError: 138891:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4034.541911] LustreError: 139319:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787564299 with bad export cookie 17833785433751786479 [ 4034.550683] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4034.556197] LustreError: 139319:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4035.088317] Lustre: server umount lustre-MDT0001 complete [ 4046.491771] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing unload_modules_local [ 4049.710556] Key type lgssc unregistered [ 4050.036099] LNet: 140549:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4050.062912] LNetError: 140549:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4050.088845] LNet: Removed LNI 192.168.206.159@tcp [ 4051.127920] Key type .llcrypt unregistered [ 4051.129783] Key type ._llcrypt unregistered [ 4075.714908] Key type ._llcrypt registered [ 4075.716520] Key type .llcrypt registered [ 4075.811725] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 4077.709305] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4077.761437] alg: No test for adler32 (adler32-zlib) [ 4078.874437] Lustre: Lustre: Build Version: 2.17.57_82_g4029ffa [ 4079.127357] LNet: Added LNI 192.168.206.159@tcp [8/256/0/180] [ 4080.792142] Key type lgssc registered [ 4082.204654] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4091.079434] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4091.097763] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4092.787939] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4096.938795] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 4103.144128] LustreError: 141968:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4103.171169] LustreError: 141968:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4106.263444] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4106.606487] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4110.382595] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 4117.003732] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4122.606859] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4128.856816] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4129.175075] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4131.312231] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4136.587655] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 4138.420832] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4144.825944] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4151.320371] Lustre: 144550:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4154.474675] Lustre: 144599:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4155.977538] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4156.893928] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4161.209716] Lustre: 144746:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4162.177627] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4163.057607] Lustre: 144844:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4167.137954] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4167.150344] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4167.170629] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4169.186564] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4169.196249] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4169.200476] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4173.053227] Lustre: server umount lustre-OST0000 complete [ 4178.403275] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4178.413405] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4178.439574] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4178.442468] Lustre: Skipped 1 previous similar message [ 4184.548060] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4184.561749] Lustre: Skipped 3 previous similar messages [ 4189.668060] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4189.678801] Lustre: Skipped 1 previous similar message [ 4191.200120] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4191.511854] Lustre: server umount lustre-MDT0000 complete [ 4193.763056] LustreError: 143268:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4193.775574] LustreError: 143268:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4195.105918] LustreError: 141947:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787564460 with bad export cookie 16372473469068278824 [ 4195.136561] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4195.443240] Lustre: server umount lustre-MDT0001 complete [ 4206.313451] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing unload_modules_local [ 4208.748619] Key type lgssc unregistered [ 4209.045107] LNet: 145909:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4209.055704] LNetError: 145909:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4209.070888] LNet: Removed LNI 192.168.206.159@tcp [ 4209.853413] Key type .llcrypt unregistered [ 4209.856255] Key type ._llcrypt unregistered [ 4229.634055] Key type ._llcrypt registered [ 4229.639061] Key type .llcrypt registered [ 4229.760697] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 4230.661195] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4230.701161] alg: No test for adler32 (adler32-zlib) [ 4231.732347] Lustre: Lustre: Build Version: 2.17.57_82_g4029ffa [ 4231.915170] LNet: Added LNI 192.168.206.159@tcp [8/256/0/180] [ 4233.552452] Key type lgssc registered [ 4234.525040] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4241.469984] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4241.494029] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4243.041399] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4247.300210] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 4253.153287] LustreError: 147305:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4253.178500] LustreError: 147305:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4255.407469] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4255.804544] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4260.403655] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 4268.927937] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4274.640920] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4281.451709] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4281.859974] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4288.064040] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4288.603280] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 4296.197062] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4301.379891] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4305.539294] Lustre: *** cfs_fail_loc=159, val=0*** [ 4321.880951] Lustre: lustre-MDT0000: Client c54a6b08-a19d-4982-8450-865a8f4c5a0e (at 192.168.206.59@tcp) reconnecting [ 4326.881647] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4326.892657] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4326.894860] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4326.923734] Lustre: Skipped 1 previous similar message [ 4328.238107] Lustre: server umount lustre-OST0000 complete [ 4332.004055] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4332.005706] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4332.023336] Lustre: Skipped 1 previous similar message [ 4332.040088] Lustre: Skipped 2 previous similar messages [ 4337.134211] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4337.142574] Lustre: Skipped 1 previous similar message [ 4338.188332] Lustre: server umount lustre-MDT0000 complete [ 4341.679081] LustreError: 147285:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787564607 with bad export cookie 14774072876745773873 [ 4341.682789] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4341.690579] LustreError: 147285:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4342.045773] Lustre: server umount lustre-MDT0001 complete [ 4352.119587] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing unload_modules_local [ 4354.806102] Key type lgssc unregistered [ 4355.092992] LNet: 150970:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4355.099201] LNetError: 150970:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4355.127305] LNet: Removed LNI 192.168.206.159@tcp [ 4355.666420] Key type .llcrypt unregistered [ 4355.668923] Key type ._llcrypt unregistered [ 4379.973964] Key type ._llcrypt registered [ 4379.975853] Key type .llcrypt registered [ 4380.083524] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 4381.383807] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4381.417629] alg: No test for adler32 (adler32-zlib) [ 4382.497747] Lustre: Lustre: Build Version: 2.17.57_82_g4029ffa [ 4382.746468] LNet: Added LNI 192.168.206.159@tcp [8/256/0/180] [ 4384.496156] Key type lgssc registered [ 4385.607055] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4393.340988] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4393.369650] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4394.790112] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4398.816804] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 4405.220474] LustreError: 152384:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4405.244255] LustreError: 152384:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4407.093407] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4407.539452] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4411.844665] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 4419.217489] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4425.331397] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4431.652640] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4432.032691] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4433.084124] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4437.882828] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 4440.255526] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4444.799875] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4463.928749] Lustre: lustre-MDT0001: evicting 776f5fbb-dc4b-4ea9-a1e9-c70c508d45dd (at 192.168.206.59@tcp) by administrative request [ 4463.939740] Lustre: Skipped 1 previous similar message [ 4494.825334] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4494.836852] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4494.852863] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4496.868311] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4496.872415] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4496.885180] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4496.895567] Lustre: Skipped 1 previous similar message [ 4498.306566] Lustre: server umount lustre-OST0000 complete [ 4501.989531] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4501.995558] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4502.002294] Lustre: Skipped 1 previous similar message [ 4502.029067] Lustre: Skipped 1 previous similar message [ 4507.107944] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4507.119133] Lustre: Skipped 1 previous similar message [ 4512.226250] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4512.233994] Lustre: Skipped 1 previous similar message [ 4516.322168] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4516.538875] Lustre: server umount lustre-MDT0000 complete [ 4517.347147] LustreError: 152380:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4517.370387] LustreError: 152380:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4519.872113] LustreError: 152366:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787564785 with bad export cookie 8471397399003373659 [ 4519.874576] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4519.887570] LustreError: 152366:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4520.249741] Lustre: server umount lustre-MDT0001 complete [ 4528.882111] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing unload_modules_local [ 4531.065650] Key type lgssc unregistered [ 4531.288694] LNet: 156462:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4531.301112] LNetError: 156462:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4531.312710] LNet: Removed LNI 192.168.206.159@tcp [ 4532.044952] Key type .llcrypt unregistered [ 4532.048724] Key type ._llcrypt unregistered [ 4595.430833] Key type ._llcrypt registered [ 4595.432953] Key type .llcrypt registered [ 4595.511345] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 4596.165272] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4596.255183] alg: No test for adler32 (adler32-zlib) [ 4597.286903] Lustre: Lustre: Build Version: 2.17.57_82_g4029ffa [ 4597.565658] LNet: Added LNI 192.168.206.159@tcp [8/256/0/180] [ 4599.217115] Key type lgssc registered [ 4600.287293] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4607.356837] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4607.388829] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4608.844916] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4613.611403] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 4619.237663] LustreError: 158950:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4619.249781] LustreError: 158950:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4623.638374] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4624.420474] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4630.087812] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 4638.816517] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4645.442473] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4652.947349] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4653.558237] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4654.611232] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4660.190444] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 4666.651907] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4668.911945] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4675.041592] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4675.047658] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4675.059000] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4679.138739] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4679.152962] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4679.153405] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4680.506142] Lustre: server umount lustre-OST0000 complete [ 4684.262630] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4684.270680] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4684.283924] Lustre: Skipped 1 previous similar message [ 4684.302398] Lustre: Skipped 2 previous similar messages [ 4689.378401] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4689.388541] Lustre: Skipped 1 previous similar message [ 4694.499888] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4694.510343] Lustre: Skipped 1 previous similar message [ 4698.080279] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4698.381884] Lustre: server umount lustre-MDT0000 complete [ 4699.617642] LustreError: 159681:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4699.637816] LustreError: 159681:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4701.599188] LustreError: 161741:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787564966 with bad export cookie 2226820260676379931 [ 4701.601214] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4701.617792] LustreError: 161741:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4702.007599] Lustre: server umount lustre-MDT0001 complete [ 4711.743242] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing unload_modules_local [ 4714.366366] Key type lgssc unregistered [ 4714.612585] LNet: 162497:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4714.617851] LNetError: 162497:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4714.627223] LNet: Removed LNI 192.168.206.159@tcp [ 4715.640978] Key type .llcrypt unregistered [ 4715.644299] Key type ._llcrypt unregistered [ 4738.187503] Key type ._llcrypt registered [ 4738.190616] Key type .llcrypt registered [ 4738.278813] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 4739.389273] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4739.458872] alg: No test for adler32 (adler32-zlib) [ 4740.600077] Lustre: Lustre: Build Version: 2.17.57_82_g4029ffa [ 4740.832106] LNet: Added LNI 192.168.206.159@tcp [8/256/0/180] [ 4742.512226] Key type lgssc registered [ 4743.527875] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4756.909363] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 05:50:21 (1787565021) [ 4771.026972] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_hostid [ 4780.109756] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 4813.933368] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4813.974607] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4815.285925] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4815.311106] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4815.413325] Lustre: lustre-MDT0000: new disk, initializing [ 4815.508912] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4815.526725] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4819.504346] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 4829.264911] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4829.379627] Lustre: 166762:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4829.437614] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4829.440928] Lustre: Skipped 1 previous similar message [ 4829.489700] Lustre: lustre-MDT0001: new disk, initializing [ 4829.557257] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4829.596303] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4829.608804] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4834.021141] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 4844.105530] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4848.775094] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4854.529269] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4854.742729] Lustre: lustre-OST0000: new disk, initializing [ 4854.746254] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4854.752597] Lustre: 168731:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4854.839996] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4860.474979] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4860.487147] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4860.612620] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4860.802833] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 4868.964674] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4881.378819] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4881.381811] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4881.385860] LustreError: Skipped 1 previous similar message [ 4881.404788] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4881.423259] Lustre: Skipped 1 previous similar message [ 4882.970417] Lustre: server umount lustre-OST0000 complete [ 4886.499874] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4886.501700] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4886.513634] Lustre: Skipped 2 previous similar messages [ 4886.534360] Lustre: Skipped 1 previous similar message [ 4891.618579] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4896.737478] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4896.749974] Lustre: Skipped 2 previous similar messages [ 4900.320341] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4900.476460] Lustre: server umount lustre-MDT0000 complete [ 4903.684447] LustreError: 166753:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787565169 with bad export cookie 12694691018880297125 [ 4903.695402] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4903.697390] LustreError: 166753:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4904.132569] Lustre: server umount lustre-MDT0001 complete [ 4913.398076] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing unload_modules_local [ 4915.866242] Key type lgssc unregistered [ 4916.180798] LNet: 170642:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4916.191473] LNetError: 170642:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4916.206881] LNet: Removed LNI 192.168.206.159@tcp [ 4917.053904] Key type .llcrypt unregistered [ 4917.056255] Key type ._llcrypt unregistered [ 4939.126202] Key type ._llcrypt registered [ 4939.128392] Key type .llcrypt registered [ 4939.275983] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 4940.446549] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4940.470434] alg: No test for adler32 (adler32-zlib) [ 4941.484627] Lustre: Lustre: Build Version: 2.17.57_82_g4029ffa [ 4941.727798] LNet: Added LNI 192.168.206.159@tcp [8/256/0/180] [ 4943.418050] Key type lgssc registered [ 4944.546202] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4951.483175] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4951.501897] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4952.933934] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4956.678421] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 4963.302108] LustreError: 172055:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4963.331276] LustreError: 172055:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4964.425716] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4964.865991] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4969.331594] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 4975.568509] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4981.276307] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4987.467985] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4987.827288] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4993.423292] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 4998.082379] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5000.152885] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5006.007585] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5006.029098] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5006.046505] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5006.056704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5006.065892] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5007.440901] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5007.488305] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5007.509957] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5007.518421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.542160] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5008.863823] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5008.889914] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5008.902754] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5008.906939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5008.924916] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5009.920325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.190513] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5011.207614] Lustre: Skipped 1 previous similar message [ 5011.234483] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5011.248496] Lustre: Skipped 1 previous similar message [ 5011.256334] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5011.262294] Lustre: Skipped 1 previous similar message [ 5011.267969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.284675] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5011.293914] Lustre: Skipped 1 previous similar message [ 5012.514428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.661899] LustreError: 172075:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5013.679168] LustreError: 172075:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5013.692940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5014.722537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5015.758429] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5015.769302] Lustre: Skipped 3 previous similar messages [ 5015.794875] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5015.814852] Lustre: Skipped 3 previous similar messages [ 5015.819268] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5015.829441] Lustre: Skipped 3 previous similar messages [ 5015.837272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5015.847934] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5015.852153] Lustre: Skipped 3 previous similar messages [ 5016.808175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.946782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5018.959854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.105607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.912667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5021.985724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.898748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5023.976353] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5023.983309] Lustre: Skipped 7 previous similar messages [ 5024.001537] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5024.008853] Lustre: Skipped 7 previous similar messages [ 5024.010942] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5024.013433] Lustre: Skipped 7 previous similar messages [ 5024.016349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.026512] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5024.029192] Lustre: Skipped 7 previous similar messages [ 5024.854374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5025.781884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5026.693170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.625155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5028.577356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.500956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5030.322694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5031.232832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5031.925642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5032.715108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.537163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.298069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.049530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.947555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.793620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5037.733512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.470157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5039.438181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.410240] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5040.421886] Lustre: Skipped 18 previous similar messages [ 5040.444134] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5040.462508] Lustre: Skipped 18 previous similar messages [ 5040.469855] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5040.478430] Lustre: Skipped 18 previous similar messages [ 5040.485142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.499149] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5040.502396] Lustre: Skipped 18 previous similar messages [ 5041.354986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.188486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.015159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.782454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.548994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.347523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.288875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.969067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.722618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.521986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.211402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.065089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.875377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.702137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.675234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.526984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.347549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.120351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.922107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.874796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.719084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.468416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.246341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.029865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.832094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.616895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.392425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.129806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.132443] LustreError: 172075:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5064.145404] LustreError: 172075:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5064.149199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.944547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.778648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.570885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.278268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.023427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.996812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.798833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.530908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.335552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.371237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.339712] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5073.348559] Lustre: Skipped 39 previous similar messages [ 5073.357991] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5073.376723] Lustre: Skipped 39 previous similar messages [ 5073.390357] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5073.399817] Lustre: Skipped 39 previous similar messages [ 5073.404343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.422917] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5073.428466] Lustre: Skipped 39 previous similar messages [ 5074.501630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.502597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.433438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.460621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.190966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.671508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.748910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.076853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.360048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.238536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.088385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.857295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.000422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.947849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.845040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.650273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.509652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.414222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.226491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.097985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.029075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.954176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.697387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.478781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.246063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.941494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.836569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.751213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.577593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.482938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.434574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.169034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.806055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.548608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.461557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.299647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.059688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.823330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.747223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.501296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.359872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.155665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.013357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.901729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.934430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.790476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.490387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.302291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.017800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.756053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.633685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.312495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.097447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.946150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.764446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.558770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.363411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.224593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.926338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.669317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.651192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.387334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.332833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.210455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.974257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.751898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.494766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.210841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.108935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.044920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.132359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.867171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.729912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.533686] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5137.537653] Lustre: Skipped 73 previous similar messages [ 5137.550400] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5137.558470] Lustre: Skipped 73 previous similar messages [ 5137.562407] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5137.566053] Lustre: Skipped 73 previous similar messages [ 5137.568687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.576552] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5137.584266] Lustre: Skipped 73 previous similar messages [ 5138.457578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.228800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.236901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.151741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.997425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.824407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.751304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.864283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.041446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.122922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.082531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.025770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.068943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.904457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.772171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.534869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.453988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.355497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.200606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.051264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.998886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.798076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.760468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.769507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.678468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.514669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.287356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.006734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.113209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.047601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.979495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.814203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.646594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.562188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.350605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.167217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.151394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.038679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.948785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.825592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.686296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.570638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.592889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.454628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.407251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.307595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.198170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.068510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.828969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.664293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.488280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.276177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.127881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.857429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.633666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.450679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.265215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.986577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.851702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.635950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.499246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.325254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.259385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.211193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.167911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.253521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.229833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.246218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.055495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.976568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.839551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.698417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.606067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.441748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.301925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.235958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.154362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.066275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.826794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.698255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.586548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.480771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.321235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.200662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.122852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.043367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.916216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.130651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.040770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.012810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.172721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.310856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.669697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.697822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.731208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.953575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.067171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.073856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.086343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.021023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.427520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.828973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.911324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.100507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.427245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.645398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.102549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.145933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.323760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.726566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.983862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.003160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.201794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.212968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.308562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.506115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.011415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.485095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.738869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.893675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.871890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.067762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.147993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.257062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.324396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.310140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.202690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.419697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.315512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.449837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.531341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.738192] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5266.746363] Lustre: Skipped 131 previous similar messages [ 5266.778704] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5266.789230] Lustre: Skipped 131 previous similar messages [ 5266.794896] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5266.800231] Lustre: Skipped 131 previous similar messages [ 5266.803476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.829527] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5266.836932] Lustre: Skipped 131 previous similar messages [ 5267.858501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.879922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.897800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.913746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.943962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.850528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.717880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.582740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.429925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.280660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.120174] LustreError: 172075:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5277.134715] LustreError: 172075:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5277.143753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.008793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.903934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.729383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.563085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.503754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.378322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.362827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.179751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.111408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.913364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.801918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.652672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.400626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.151336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.034590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.066773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.060057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.997777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.713599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.649700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.620487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.485488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.468134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.439723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.617467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.539761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.852106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.172973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.379526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.205409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.291966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.084503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.005535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.066672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.958889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.927800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.781081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.820320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.594367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.273034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.062256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.001676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.919959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.830930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.636084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.489364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.320958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.178569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.951100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.881509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.698473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.636338] LustreError: 172075:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5324.656601] LustreError: 172075:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5324.666133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.546806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.388173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.202400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.083649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.054710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.386828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.297361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.351753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.210961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.926770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.654332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.444847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.150591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.955613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.786990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.921483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.848853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.784257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.763728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.766599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.933595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.848306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.877254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.764814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.730758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.667113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.792443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.722872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.517494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.329320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.305866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.947232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.711434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.396416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.172391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.986883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.811402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.573823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.272355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.979352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.870306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.678248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.394099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.148128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.020246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.847179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.632232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.429449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.245587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.097051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.962382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.814316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.683038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.556752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.285286] LustreError: 172075:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5372.296433] LustreError: 172075:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5372.300314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.033842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.964267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.943970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.874354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.954279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.954315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.026870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.644831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.465455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.680236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.625622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.359128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.181591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.020939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.840531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.721900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.446862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.539974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.617630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.519608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.522796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.656933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.579706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.544135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.568795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.497204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.385166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.270767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.099283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.161068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.953366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.910491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.837950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.843156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.679960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.724695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.619174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.770993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.484661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.503506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.817054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.467333] LustreError: 172075:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5411.472481] LustreError: 172075:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5411.480631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.316037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.255418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.655606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.759604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.965938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.972161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.116793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.096403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.120897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.010028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.853204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.822299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.834153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.020084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.015963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.200807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.023189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.998745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.979646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.889409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.800263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.864981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.731800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.663250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.584855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.504644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.384572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.194509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.132019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.122285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.175481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.002878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.795956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.779453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.927275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.762233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.772852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.658384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.674665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.487743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.238460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.153981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.103938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.900590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.731979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.426335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.429698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.181870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.965445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.707981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.553549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.268571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.936247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.531357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.293168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.927255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.818831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.653648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.587281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.345470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.097641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.903059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.674326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.380546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.086331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.856671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.619200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.556114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.490889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.200141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.927115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.765502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.735314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.751215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.541682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.510517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.419188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.234598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.019746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.757132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.573754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.458916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.330949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.097084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.987597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.867500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.792606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.806038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.734208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.578621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.432137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.282876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.284911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.120828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.093583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.941941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.929557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.790734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.734254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.633811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.666831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.571619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.414032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.344483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.363415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.376675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.473421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.647398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.790100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.951862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.949810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.982338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.934966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.767525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.680284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.487995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.308324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.207862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.118762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.967372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.841195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.838321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.703416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.654462] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5523.664882] Lustre: Skipped 283 previous similar messages [ 5523.679357] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5523.687686] Lustre: Skipped 283 previous similar messages [ 5523.691996] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5523.699183] Lustre: Skipped 283 previous similar messages [ 5523.702277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.710611] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5523.715532] Lustre: Skipped 283 previous similar messages [ 5524.495521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.370532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.267384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.075720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.113468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.208290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.175709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.949237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.917215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.838470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.666236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.615828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.589725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.696500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.553662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.519959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.461432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.334261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.193621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.080649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.044368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.008972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.084378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.035980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.979811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.864218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.676410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.534816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.737307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.508476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.375481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.290669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.085540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.838053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.693026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.504044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.276879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.209350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.282556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.433259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.251107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.273471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.228689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.223330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.119808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.977795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.846180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.649949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.476686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.443035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.324815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.250304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.086566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.886200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.837143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.810851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.662535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.439235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.128817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.998588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.869257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.860384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.735997] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.626722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.409036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.281254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.236628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.072884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.889615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.650726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.328445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.066325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.849240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.652710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.472048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.323704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.132240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.092654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.998261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.950028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.801778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.842644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.759405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.533587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.307671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.257910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.897758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.590396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.373462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.088805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.896805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.794621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.580653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.288595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.127418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.843540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.625597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.347588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.130194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.971221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.692538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.460572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.181993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.909937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.695571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.507797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.309365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.135417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.771550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.426784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.197880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.070094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.049070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.832760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.754203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.606763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.564378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.458884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.373649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.231142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.193605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.020561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.937100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.796756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.666694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.507542] LustreError: 172075:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5632.516482] LustreError: 172075:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5632.525961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.455644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.229924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.165708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.155333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.036866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.883352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.731773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.641230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.482731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.256980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.113591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.001129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.744077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.568725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.287665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.133957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.919864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.630742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.414755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.151766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.886474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.685517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.550465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.488629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.444246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.234784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.092158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.895567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.756698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.617733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.431818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.382261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.215455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.145502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.020679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.774752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.512621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.368070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.284400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.267496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.332253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.287376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.334587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.234743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.162321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.088760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.011688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.879485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.879739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.697306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.685663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.562128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.485567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.272469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.199753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.178485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.957989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.763668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.567670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.377198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.290344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.023843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.866171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.690400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.473380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.355548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.174090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.064615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.852913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.540827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.339625] LustreError: 172075:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5693.348764] LustreError: 172075:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5693.349485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.041883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.895425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.665694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.450897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.074622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.932741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.672511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.491289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.201536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.936347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.719458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.552362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.232984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.976595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.718805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.560689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.424948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.335354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.129344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.217767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.394034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.245696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.159793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.954508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.821733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.643571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.617627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.623372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.378827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.169808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.989196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.706254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.482335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.206182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.024944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.981045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.982311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.905537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.744687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.674070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.545341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.478409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.286356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.146151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.022569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.786601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.757703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.857276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.826025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.854729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.816213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.864435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.841241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.816070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.688933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.628393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.471289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.578195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.471689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.807727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.951240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.647133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.865044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.970801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.844982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.907482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.818347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.972638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.033626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.932120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.900538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.714756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.679788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.562967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.312822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.006254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.983575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.017461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.051503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.904987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.874527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.137615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.150088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.228701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.120831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.020961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.895299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.883248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.666517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.506675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.401355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.393263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.332306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.226194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.206829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.325385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.356622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.217613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.986454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.857397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.861752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.648260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.778823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.803938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.761324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.696134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.479564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.210806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.007454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.828135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.789343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.638157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.543298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.394338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.162051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.151837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.936436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.802495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.471587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.240749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.942183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.729910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.639921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.593424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.452543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.246941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.046602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.011888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.734303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.585258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.299718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.093660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.798995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.459720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.317550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.118376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.995624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.787829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.698797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.633181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.518762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.283791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.311186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.294670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.313253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.058384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.922029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.224561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.400942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.415675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.535927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.733838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.644994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.613971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.473595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.325737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.367337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.276478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.062324] LustreError: 172075:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5836.068971] LustreError: 172075:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5836.092431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.941406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.037342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.873245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.779679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.591652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.596917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.390237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.398689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.449295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.300347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.208941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.175712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.833430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.729681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.638672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.533050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.244209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.150529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.255561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.230158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.125956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.142736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.000444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.085739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.977250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.977813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.205772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.617834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.859968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.036900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.116240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.112352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.002148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.854540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.931667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.316728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.737300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.262091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.470467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.369812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.384763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.424613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.301696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.314518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.339976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.209147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.964072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.649321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.422187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.376897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.212803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.221757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.927488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.880465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.737821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.663448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.366683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.074027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.874322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.772202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.707489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.610830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.298186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.113387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.754985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.542551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.353035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.094814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.853761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.577625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.351231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.352267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.363891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.165902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.091560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.844049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.701072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.536676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6618.085776] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6618.092895] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6618.097912] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6618.114243] Lustre: Skipped 435 previous similar messages [ 6622.905336] Lustre: server umount lustre-OST0000 complete [ 6628.323881] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6628.331194] Lustre: Skipped 1 previous similar message [ 6633.442229] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6633.449259] Lustre: Skipped 2 previous similar messages [ 6633.825878] Lustre: server umount lustre-MDT0000 complete [ 6638.100268] LustreError: 173897:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787566903 with bad export cookie 17187606486541932722 [ 6638.116543] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6638.125489] LustreError: 173897:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6638.542305] Lustre: server umount lustre-MDT0001 complete [ 6649.530820] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing unload_modules_local [ 6652.588717] Key type lgssc unregistered [ 6652.932461] LNet: 227262:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6652.939754] LNetError: 227262:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6652.961522] LNet: Removed LNI 192.168.206.159@tcp [ 6653.883195] Key type .llcrypt unregistered [ 6653.887650] Key type ._llcrypt unregistered [ 6679.905976] Key type ._llcrypt registered [ 6679.910227] Key type .llcrypt registered [ 6680.016227] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 6681.199324] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6681.417769] alg: No test for adler32 (adler32-zlib) [ 6682.752562] Lustre: Lustre: Build Version: 2.17.57_82_g4029ffa [ 6683.113260] LNet: Added LNI 192.168.206.159@tcp [8/256/0/180] [ 6684.864745] Key type lgssc registered [ 6685.967441] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6693.477765] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6693.505756] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6694.997582] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6699.113283] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 6705.124399] LustreError: 228680:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6705.136087] LustreError: 228680:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 6706.774584] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6707.067158] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6711.433696] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 6719.570138] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6725.077995] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6732.052915] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6732.527831] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6739.691158] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 6739.722392] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 6747.160200] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6755.829181] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6757.210599] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6757.226759] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6757.233988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6757.246808] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6757.253387] LustreError: 228700:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 6759.989169] Lustre: lustre-OST0000: Not available for connect from 192.168.206.59@tcp (stopping) [ 6762.472503] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6762.475469] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6762.500471] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6762.505561] Lustre: Skipped 1 previous similar message [ 6765.084777] Lustre: lustre-OST0000: Not available for connect from 192.168.206.59@tcp (stopping) [ 6767.585791] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6767.597280] Lustre: Skipped 1 previous similar message [ 6772.709928] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6772.719154] Lustre: Skipped 2 previous similar messages [ 6773.216200] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6773.309257] Lustre: server umount lustre-OST0000 complete [ 6777.304282] LustreError: 228700:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 6806.857813] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6807.376273] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6814.017735] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 6822.753435] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6827.727229] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 3 sec [ 6827.826982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6827.857274] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6827.858013] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6827.888321] Lustre: Skipped 1 previous similar message [ 6833.540151] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff95ade0631800.ost_server_uuid 50 [ 6834.762021] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff95ade0631800.ost_server_uuid in FULL state after 0 sec [ 6843.378381] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6843.386316] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6843.400538] Lustre: Skipped 2 previous similar messages [ 6843.424154] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6844.548577] Lustre: server umount lustre-OST0000 complete [ 6848.490330] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6848.516132] Lustre: Skipped 1 previous similar message [ 6862.304119] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6862.552609] Lustre: server umount lustre-MDT0000 complete [ 6862.822724] LustreError: 228681:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6862.844634] LustreError: 228681:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6865.739280] LustreError: 228662:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787567131 with bad export cookie 13818799946177634977 [ 6865.771749] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6866.214373] Lustre: server umount lustre-MDT0001 complete [ 6875.992286] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing unload_modules_local [ 6878.733912] Key type lgssc unregistered [ 6879.021653] LNet: 233896:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6879.030917] LNetError: 233896:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6880.105697] LNet: Removed LNI 192.168.206.159@tcp [ 6880.992900] Key type .llcrypt unregistered [ 6880.996690] Key type ._llcrypt unregistered [ 6911.067715] Key type ._llcrypt registered [ 6911.072764] Key type .llcrypt registered [ 6911.196983] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 6912.673335] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6912.708545] alg: No test for adler32 (adler32-zlib) [ 6913.823538] Lustre: Lustre: Build Version: 2.17.57_82_g4029ffa [ 6914.123758] LNet: Added LNI 192.168.206.159@tcp [8/256/0/180] [ 6915.869654] Key type lgssc registered [ 6917.511701] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6928.328069] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6928.346828] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6929.969987] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6935.516249] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 6940.131239] LustreError: 235631:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6940.170710] LustreError: 235631:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 6945.249802] LustreError: 235630:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6945.572140] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6946.005828] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6951.126634] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 6954.059427] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 6960.060642] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6960.299358] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6966.965616] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 6975.671948] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 6976.412426] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6976.592689] Lustre: lustre-OST0001: new disk, initializing [ 6976.597916] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 6976.622587] Lustre: 237939:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6976.757986] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6982.723887] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 6982.735397] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 6982.817604] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 6983.022486] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 6994.652965] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7005.952584] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7018.977804] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7018.990763] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7018.993558] Lustre: Skipped 3 previous similar messages [ 7019.004341] Lustre: Skipped 2 previous similar messages [ 7019.247959] Lustre: server umount lustre-MDT0000 complete [ 7022.378537] LustreError: 237938:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787567287 with bad export cookie 11197094560098733484 [ 7022.382572] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7022.404542] LustreError: 237938:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7024.110703] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7024.110856] LustreError: 235625:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7024.111953] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7024.111959] Lustre: Skipped 1 previous similar message [ 7024.121953] Lustre: Skipped 1 previous similar message [ 7024.131452] LustreError: 235625:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7024.754959] Lustre: server umount lustre-MDT0001 complete [ 7028.043573] Lustre: server umount lustre-OST0000 complete [ 7031.303478] Lustre: server umount lustre-OST0001 complete [ 7053.880608] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 7065.789964] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7066.092322] LustreError: 241042:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7066.148209] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7071.137177] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 7071.211182] LustreError: 241043:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7081.407762] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7081.454116] LustreError: 241043:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7081.509945] LustreError: 241043:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7082.452406] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7088.403545] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 7091.128491] Lustre: 242185:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7097.554546] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7104.197060] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 7107.178927] LustreError: 242539:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7111.291625] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49760) [ 7112.968080] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7113.229407] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7113.233632] Lustre: Skipped 1 previous similar message [ 7118.340034] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 7119.407716] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 7126.889237] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7130.029558] Lustre: 244054:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7143.916687] Lustre: mylustre-MDT0000-lwp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7143.919883] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7143.920539] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7143.920544] Lustre: Skipped 1 previous similar message [ 7143.932195] Lustre: Skipped 2 previous similar messages [ 7146.568871] Lustre: server umount mylustre-MDT0000 complete [ 7149.037786] LustreError: 241776:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7149.056845] LustreError: 241776:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 7150.193953] LustreError: 241017:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787567415 with bad export cookie 11197094560098736277 [ 7150.201439] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7150.210294] LustreError: 241017:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7150.489291] Lustre: server umount mylustre-MDT0001 complete [ 7165.344601] Lustre: server umount mylustre-OST0000 complete [ 7179.406323] Lustre: server umount mylustre-OST0001 complete [ 7199.367206] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 7208.710715] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7209.200130] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7214.218177] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 7214.569191] LustreError: 246364:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7214.598300] LustreError: 246364:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7223.473145] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7223.777626] Lustre: tfs-MDT0001: Not available for connect from 0@lo (not set up) [ 7223.787967] Lustre: Skipped 3 previous similar messages [ 7229.265308] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 7232.725615] Lustre: 247506:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7241.365654] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7250.329778] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 7254.396694] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49792) [ 7260.090519] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7265.804891] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 7267.258683] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 7274.350924] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7277.742814] Lustre: 249379:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7284.196112] Lustre: tfs-MDT0000-lwp-OST0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7284.201904] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7284.210525] Lustre: Skipped 4 previous similar messages [ 7284.221320] Lustre: Skipped 3 previous similar messages [ 7286.314927] Lustre: server umount tfs-MDT0000 complete [ 7289.441206] LustreError: 246340:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787567554 with bad export cookie 11197094560098738167 [ 7289.448181] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7289.461967] LustreError: 246340:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7289.824388] Lustre: server umount tfs-MDT0001 complete [ 7303.878922] Lustre: server umount tfs-OST0000 complete [ 7317.850832] Lustre: server umount tfs-OST0001 complete [ 7338.179353] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 7346.471474] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7346.955726] LustreError: 251629:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7346.976479] LustreError: 251629:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 10 previous similar messages [ 7347.042043] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7347.050290] Lustre: Skipped 3 previous similar messages [ 7351.337526] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 7358.821204] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7363.719110] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 7366.428858] Lustre: 252770:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7372.792854] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7379.260244] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 7387.634619] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7392.003512] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 7392.030029] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49824) [ 7393.640382] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 7401.391833] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7405.423569] Lustre: 254640:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7412.784581] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 06:34:36 (1787567676) [ 7431.137243] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7431.152054] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7431.183258] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7434.779150] Lustre: server umount lustre-MDT0000 complete [ 7438.559431] LustreError: 251606:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787567703 with bad export cookie 11197094560098739826 [ 7438.566721] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7438.585275] LustreError: 251606:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7443.428393] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7443.464099] Lustre: Skipped 3 previous similar messages [ 7445.167346] Lustre: server umount lustre-MDT0001 complete [ 7454.769781] Lustre: server umount lustre-OST0000 complete [ 7464.491850] Lustre: server umount lustre-OST0001 complete [ 7471.202675] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_hostid [ 7480.671863] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 7521.046317] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7521.230838] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7521.247494] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7521.309513] Lustre: lustre-MDT0000: new disk, initializing [ 7521.369986] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 7521.374813] Lustre: Skipped 3 previous similar messages [ 7521.386121] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7525.961547] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 7535.973961] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7536.117731] Lustre: 257822:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7536.264199] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7536.273115] Lustre: Skipped 1 previous similar message [ 7536.412978] Lustre: lustre-MDT0001: new disk, initializing [ 7536.586087] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7536.603570] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7541.312813] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 7553.203242] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7553.490220] Lustre: lustre-OST0000: new disk, initializing [ 7553.493904] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7553.499478] Lustre: 259403:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7555.089397] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7555.100524] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7555.238025] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7560.389238] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 7572.156335] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7572.430983] Lustre: lustre-OST0001: new disk, initializing [ 7572.437864] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7572.449568] Lustre: 260275:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7574.088719] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7574.107312] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7574.236136] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7579.678956] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 7593.494937] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 06:37:37 (1787567857) [ 7609.662295] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 06:37:53 (1787567873) [ 7625.710088] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7625.732161] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7625.738873] Lustre: Skipped 2 previous similar messages [ 7625.749177] Lustre: Skipped 6 previous similar messages [ 7631.501656] Lustre: server umount lustre-OST0000 complete [ 7635.938573] LustreError: 259389:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7635.962010] LustreError: 259389:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 10 previous similar messages [ 7642.152711] Lustre: server umount lustre-MDT0000 complete [ 7646.822700] LustreError: 258614:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787567912 with bad export cookie 11197094560098741387 [ 7646.833659] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7646.840840] LustreError: 258614:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7653.406299] Lustre: server umount lustre-MDT0001 complete [ 7665.268955] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing unload_modules_local [ 7740.892149] Lustre: server umount lustre-OST0001 complete [ 7748.237110] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_hostid [ 7766.593811] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 7807.910172] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7808.108550] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7808.127791] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7808.194953] Lustre: lustre-MDT0000: new disk, initializing [ 7808.309145] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7808.316863] Lustre: Skipped 3 previous similar messages [ 7808.348924] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7813.693277] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 7825.507860] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7825.727562] Lustre: 265134:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7825.813855] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7825.817251] Lustre: Skipped 1 previous similar message [ 7825.894892] Lustre: lustre-MDT0001: new disk, initializing [ 7826.011959] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7826.023263] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7830.748612] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 7841.275270] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7848.005488] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7855.917789] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7856.238602] Lustre: 267106:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7857.771902] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7857.971983] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7862.830256] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 7874.148348] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7883.749889] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7883.757228] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7883.778079] Lustre: Skipped 5 previous similar messages [ 7883.787017] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7883.796374] Lustre: Skipped 11 previous similar messages [ 7886.969875] Lustre: server umount lustre-OST0000 complete [ 7892.457171] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7897.799259] Lustre: server umount lustre-MDT0000 complete [ 7902.951913] LustreError: 265129:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787568168 with bad export cookie 11197094560098742857 [ 7902.959792] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7902.969192] LustreError: 265129:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7903.579109] Lustre: server umount lustre-MDT0001 complete [ 7917.280428] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing unload_modules_local [ 7921.039476] Key type lgssc unregistered [ 7921.289622] LNet: 268869:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7921.296393] LNetError: 268869:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7921.307544] LNet: Removed LNI 192.168.206.159@tcp [ 7922.355303] Key type .llcrypt unregistered [ 7922.362052] Key type ._llcrypt unregistered [ 7951.715887] Key type ._llcrypt registered [ 7951.718613] Key type .llcrypt registered [ 7951.849981] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 7953.026294] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7953.096297] alg: No test for adler32 (adler32-zlib) [ 7954.247856] Lustre: Lustre: Build Version: 2.17.57_82_g4029ffa [ 7954.502430] LNet: Added LNI 192.168.206.159@tcp [8/256/0/180] [ 7956.136148] Key type lgssc registered [ 7957.509384] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7966.592275] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7966.638769] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7968.255783] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7974.157393] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 7978.488294] LustreError: 270331:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7978.531493] LustreError: 270331:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7983.592942] LustreError: 270332:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7984.717242] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7984.992717] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7989.605426] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 7998.526242] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8003.869155] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8011.289991] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8011.892298] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8012.931911] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8019.227791] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 8029.207482] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8030.325380] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8033.761558] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8033.771069] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8033.807567] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8035.814830] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8035.818418] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8035.820681] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8035.881359] Lustre: Skipped 1 previous similar message [ 8039.115970] Lustre: server umount lustre-OST0000 complete [ 8046.063906] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8046.107207] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8050.291800] Lustre: server umount lustre-MDT0000 complete [ 8051.179563] LustreError: 270327:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8051.204069] LustreError: 270327:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8055.017648] LustreError: 270313:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787568320 with bad export cookie 5442743101943806259 [ 8055.031248] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8055.034228] LustreError: 270313:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8055.716612] Lustre: server umount lustre-MDT0001 complete [ 8067.222302] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing unload_modules_local [ 8070.127050] Key type lgssc unregistered [ 8070.455466] LNet: 273921:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8070.473874] LNetError: 273921:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8070.492101] LNet: Removed LNI 192.168.206.159@tcp [ 8071.330395] Key type .llcrypt unregistered [ 8071.333834] Key type ._llcrypt unregistered [ 8088.834365] Key type ._llcrypt registered [ 8088.836621] Key type .llcrypt registered [ 8088.934985] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 8090.134602] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8090.192118] alg: No test for adler32 (adler32-zlib) [ 8091.275598] Lustre: Lustre: Build Version: 2.17.57_82_g4029ffa [ 8091.564212] LNet: Added LNI 192.168.206.159@tcp [8/256/0/180] [ 8093.248176] Key type lgssc registered [ 8094.649799] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8104.201364] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8104.214924] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8119.712724] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8124.832541] LustreError: 275119:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.159@tcp: failed processing log, type 4: rc = -110 [ 8151.521746] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8159.569666] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 8167.831783] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8171.091846] Lustre: server umount lustre-OST0000 complete [ 8190.141445] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing unload_modules_local [ 8194.297616] Key type lgssc unregistered [ 8194.635772] LNet: 276754:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8194.647478] LNetError: 276754:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8194.668930] LNet: Removed LNI 192.168.206.159@tcp [ 8195.936613] Key type .llcrypt unregistered [ 8195.940778] Key type ._llcrypt unregistered [ 8246.965442] Key type ._llcrypt registered [ 8246.968536] Key type .llcrypt registered [ 8247.061890] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 8248.157284] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8248.196412] alg: No test for adler32 (adler32-zlib) [ 8249.214475] Lustre: Lustre: Build Version: 2.17.57_82_g4029ffa [ 8249.460583] LNet: Added LNI 192.168.206.159@tcp [8/256/0/180] [ 8251.176151] Key type lgssc registered [ 8252.636920] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8266.928696] loop0: detected capacity change from 0 to 204800000 [ 8267.193424] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8267.213784] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8267.817561] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8268.954204] loop0: detected capacity change from 0 to 204800000 [ 8269.178217] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8269.188829] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8269.773776] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8270.898805] loop0: detected capacity change from 0 to 204800000 [ 8271.138256] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8271.153512] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8271.314489] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8272.565467] loop0: detected capacity change from 0 to 204800000 [ 8272.792723] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8272.807035] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8272.962873] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8274.083975] loop0: detected capacity change from 0 to 204800000 [ 8274.116202] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8278.817361] loop0: detected capacity change from 0 to 204800000 [ 8278.868457] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8283.518436] loop0: detected capacity change from 0 to 204800000 [ 8283.581492] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8287.851269] loop0: detected capacity change from 0 to 204800000 [ 8287.907595] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8292.320264] loop0: detected capacity change from 0 to 204800000 [ 8292.432738] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8292.550986] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8292.658246] Lustre: 280171:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8292.677504] Lustre: 280171:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8292.712036] Lustre: 280171:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8292.739416] Lustre: 280171:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8292.773712] Lustre: 280171:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8292.801750] Lustre: 280171:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8292.830949] Lustre: 280171:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8292.853296] Lustre: 280171:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8292.883537] Lustre: 280171:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8292.905644] Lustre: 280171:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8292.926402] Lustre: 280171:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8292.948271] Lustre: 280171:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8292.992103] Lustre: 280171:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8293.022896] Lustre: 280171:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8299.560795] Lustre: server umount MGS complete [ 8300.571839] loop0: detected capacity change from 0 to 204800000 [ 8300.705211] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8301.550105] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8301.606651] Lustre: lustre-MDT0000: Aborting client recovery [ 8302.588117] loop1: detected capacity change from 0 to 204800000 [ 8302.781915] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8302.924580] Lustre: 280603:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8302.952639] Lustre: 280603:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8302.982119] Lustre: 280603:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8303.010714] Lustre: 280603:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8303.020066] Lustre: 280603:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8303.054898] Lustre: 280603:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8303.086579] Lustre: 280603:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8303.136879] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8303.424390] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8303.476418] Lustre: lustre-MDT0001: Aborting client recovery [ 8304.653325] loop2: detected capacity change from 0 to 204800000 [ 8304.946609] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8305.019810] Lustre: 280684:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8305.030942] Lustre: 280684:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8305.048597] Lustre: 280684:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8305.063280] Lustre: 280684:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8305.083514] Lustre: 280684:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8305.092916] Lustre: 280684:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8305.105600] Lustre: 280684:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8305.234950] Lustre: Found index 0 for lustre-OST0000, updating log [ 8305.319388] Lustre: 280702:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8305.506069] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8305.524687] Lustre: lustre-OST0000: Aborting recovery [ 8306.587857] loop3: detected capacity change from 0 to 204800000 [ 8306.793976] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8306.937567] Lustre: 280768:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8306.968488] Lustre: 280768:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8306.988094] Lustre: 280768:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8307.037337] Lustre: 280768:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8307.056236] Lustre: 280768:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8307.086597] Lustre: 280768:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8307.108074] Lustre: 280768:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8307.130311] Lustre: Found index 1 for lustre-OST0001, updating log [ 8307.178127] Lustre: 280774:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8307.276782] Lustre: lustre-OST0001: Aborting recovery [ 8311.366469] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8311.407662] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8311.430252] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8311.456281] LustreError: 280547:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 8311.495060] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8313.448348] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8313.448653] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8316.520182] LustreError: 280514:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 6412us: rc = -2 [ 8317.040753] LustreError: 280512:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 5028us: rc = -2 [ 8317.084906] LustreError: 280512:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 15 previous similar messages [ 8318.076616] LustreError: 280514:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 516us: rc = -2 [ 8318.106834] LustreError: 280514:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 26 previous similar messages [ 8334.310762] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8334.313206] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8334.314469] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8334.334248] Lustre: Skipped 1 previous similar message [ 8337.220631] Lustre: server umount lustre-MDT0000 complete [ 8338.400518] LustreError: 280773:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787568603 with bad export cookie 266164671894048776 [ 8338.428858] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8339.426385] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8339.429407] LustreError: 281162:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8339.440938] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8339.440946] Lustre: Skipped 3 previous similar messages [ 8339.453995] Lustre: Skipped 3 previous similar messages [ 8339.504856] LustreError: 281162:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8343.457343] LustreError: 280511:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8343.459873] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8343.507890] Lustre: Skipped 2 previous similar messages [ 8345.773154] Lustre: server umount lustre-MDT0001 complete [ 8353.157309] Lustre: server umount lustre-OST0000 complete [ 8360.776939] Lustre: server umount lustre-OST0001 complete [ 8371.328705] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 06:50:35 (1787568635) [ 8412.273650] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_hostid [ 8423.265649] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 8461.720809] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8462.094808] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8462.172555] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8462.290184] Lustre: lustre-MDT0000: new disk, initializing [ 8462.427152] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8462.441192] Lustre: Skipped 1 previous similar message [ 8462.483696] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8467.598305] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 8479.856744] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8480.082948] Lustre: 284842:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8480.199515] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8480.207982] Lustre: Skipped 1 previous similar message [ 8480.379341] Lustre: lustre-MDT0001: new disk, initializing [ 8480.551749] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8480.609850] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8480.624723] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8485.944265] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 8497.734465] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8504.577275] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8512.994570] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8513.253631] Lustre: lustre-OST0000: new disk, initializing [ 8513.257101] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8513.261664] Lustre: 286811:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8513.331500] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8518.771844] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8518.791737] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8518.983858] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8522.409704] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 8535.604483] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8541.415281] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8554.206722] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8563.749491] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8572.635154] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8621.546285] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8621.553640] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8621.558302] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8621.575227] Lustre: Skipped 1 previous similar message [ 8624.899257] Lustre: server umount lustre-OST0000 complete [ 8631.778612] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8631.784387] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8631.800021] Lustre: Skipped 1 previous similar message [ 8631.823393] Lustre: Skipped 2 previous similar messages [ 8635.769365] Lustre: server umount lustre-MDT0000 complete [ 8636.898951] LustreError: 287242:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8636.920185] LustreError: 287242:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8640.471720] LustreError: 288463:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787568905 with bad export cookie 266164671894051478 [ 8640.475090] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8640.483646] LustreError: 288463:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 8640.894796] Lustre: server umount lustre-MDT0001 complete [ 8651.646532] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8657.343387] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 8662.988341] Lustre: server umount MGS complete [ 8675.214672] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8675.697363] LustreError: 290829:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8675.724647] LustreError: 290842:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8675.816307] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8680.874909] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 8680.931637] LustreError: 290830:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8689.329284] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8694.827497] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 8702.685447] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8708.275045] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8715.315976] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8715.565973] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8715.572495] Lustre: Skipped 1 previous similar message [ 8722.969775] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 8731.451663] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8765.921221] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8765.928977] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8765.941352] LustreError: Skipped 1 previous similar message [ 8765.946206] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8765.958277] Lustre: Skipped 1 previous similar message [ 8765.984773] Lustre: Skipped 1 previous similar message [ 8770.021896] Lustre: server umount lustre-OST0000 complete [ 8776.175851] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8776.192041] Lustre: Skipped 1 previous similar message [ 8780.377821] Lustre: server umount lustre-MDT0000 complete [ 8781.288558] LustreError: 291558:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8781.313693] LustreError: 291558:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8784.482138] LustreError: 293555:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787569049 with bad export cookie 266164671894053431 [ 8784.483538] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8784.496887] LustreError: 293555:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8784.903644] Lustre: server umount lustre-MDT0001 complete [ 8798.838901] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing unload_modules_local [ 8802.447715] Key type lgssc unregistered [ 8802.772572] LNet: 294777:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8802.786788] LNetError: 294777:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8802.822254] LNet: Removed LNI 192.168.206.159@tcp [ 8803.825276] Key type .llcrypt unregistered [ 8803.829226] Key type ._llcrypt unregistered [ 8847.649775] Key type ._llcrypt registered [ 8847.654597] Key type .llcrypt registered [ 8847.781078] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_hostid [ 8864.761390] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 8866.028941] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8866.151897] alg: No test for adler32 (adler32-zlib) [ 8867.302145] Lustre: Lustre: Build Version: 2.17.57_82_g4029ffa [ 8867.685379] LNet: Added LNI 192.168.206.159@tcp [8/256/0/180] [ 8869.353171] Key type lgssc registered [ 8870.518844] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8906.240936] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8906.290919] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8907.489204] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8907.533033] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8907.620548] Lustre: lustre-MDT0000: new disk, initializing [ 8907.702418] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8907.726874] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8912.693136] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 8924.248833] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8924.391059] Lustre: 298738:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8924.455190] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8924.458931] Lustre: Skipped 1 previous similar message [ 8924.534356] Lustre: lustre-MDT0001: new disk, initializing [ 8924.672768] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8924.755132] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8924.774326] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8929.746364] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 8940.846569] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8947.500218] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8956.329683] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8956.630989] Lustre: lustre-OST0000: new disk, initializing [ 8956.634500] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8956.639351] Lustre: 300709:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8956.705884] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8963.908304] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 8966.206762] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8966.215184] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8966.382670] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8974.240989] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8981.424832] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8991.191293] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8994.974978] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9000.630437] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9038.308138] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9038.309488] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9038.315837] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9038.317219] LustreError: Skipped 1 previous similar message [ 9038.351770] Lustre: Skipped 1 previous similar message [ 9043.428858] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9043.432877] Lustre: Skipped 2 previous similar messages [ 9048.557237] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9048.561512] Lustre: Skipped 1 previous similar message [ 9050.080152] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9050.226249] Lustre: server umount lustre-OST0000 complete [ 9057.761172] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9057.782429] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9057.815742] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9058.788271] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9060.380532] Lustre: server umount lustre-MDT0000 complete [ 9063.945995] LustreError: 300708:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787569329 with bad export cookie 13056506437845281795 [ 9063.954842] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9063.955531] LustreError: 300708:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9064.516412] Lustre: server umount lustre-MDT0001 complete [ 9073.397091] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9078.117777] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 9082.956975] Lustre: server umount MGS complete [ 9095.018582] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9095.562757] LustreError: 304488:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9095.583615] LustreError: 304488:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9095.602400] LustreError: 304501:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9095.696048] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9100.095874] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 9100.770946] LustreError: 304489:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9104.872179] LustreError: 304489:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9109.864024] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9109.985318] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 9109.993544] Lustre: Skipped 2 previous similar messages [ 9110.195905] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9114.641816] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 9122.294755] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9128.407920] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9136.836618] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9137.196071] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9144.398961] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 9152.655642] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9186.273124] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9186.275800] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9186.287743] LustreError: Skipped 1 previous similar message [ 9186.290144] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9186.315324] Lustre: Skipped 1 previous similar message [ 9186.329133] Lustre: Skipped 1 previous similar message [ 9191.919920] Lustre: server umount lustre-OST0000 complete [ 9196.514202] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9196.528628] Lustre: Skipped 1 previous similar message [ 9201.570419] Lustre: server umount lustre-MDT0000 complete [ 9201.636150] LustreError: 305528:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9201.649374] LustreError: 305528:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9205.136030] LustreError: 307215:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787569470 with bad export cookie 13056506437845283629 [ 9205.148229] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9205.158638] LustreError: 307215:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9205.623240] Lustre: server umount lustre-MDT0001 complete [ 9218.220417] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing unload_modules_local [ 9221.297282] Key type lgssc unregistered [ 9221.609146] LNet: 308439:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9221.624786] LNetError: 308439:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9221.644284] LNet: Removed LNI 192.168.206.159@tcp [ 9222.646362] Key type .llcrypt unregistered [ 9222.650487] Key type ._llcrypt unregistered [ 9246.144760] Key type ._llcrypt registered [ 9246.147219] Key type .llcrypt registered [ 9246.272599] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_hostid [ 9264.313243] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 9265.235814] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9265.308324] alg: No test for adler32 (adler32-zlib) [ 9266.662778] Lustre: Lustre: Build Version: 2.17.57_82_g4029ffa [ 9266.970435] LNet: Added LNI 192.168.206.159@tcp [8/256/0/180] [ 9268.648976] Key type lgssc registered [ 9270.163271] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9298.455884] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [ 9314.923988] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9314.980659] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9316.272992] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9316.330446] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9316.440699] Lustre: lustre-MDT0000: new disk, initializing [ 9316.532933] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9316.547075] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9321.217704] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 9332.642238] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9332.942964] Lustre: lustre-OST0000: new disk, initializing [ 9332.950673] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9332.955385] Lustre: Skipped 1 previous similar message [ 9332.962725] Lustre: 312502:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9333.045419] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9339.848542] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [ 9342.579374] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9342.598764] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9342.630669] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [10105.316724] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10105.340165] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10109.391715] Lustre: server umount lustre-MDT0000 complete [10112.778874] LustreError: 311688:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787570378 with bad export cookie 13911572125100197029 [10112.789544] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10112.791687] LustreError: 311688:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10112.952551] Lustre: server umount lustre-OST0000 complete [10137.137557] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_hostid [10147.044199] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [10176.368113] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10176.598475] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10176.631783] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10176.717421] Lustre: lustre-MDT0000: new disk, initializing [10176.807415] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10176.820684] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10180.722846] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [10190.791365] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10190.907691] Lustre: 316828:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10190.961761] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10190.965575] Lustre: Skipped 1 previous similar message [10191.038388] Lustre: lustre-MDT0001: new disk, initializing [10191.125433] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10191.156602] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10191.163822] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10196.052952] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [10205.164345] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10211.162544] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10218.559140] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10218.794412] Lustre: lustre-OST0000: new disk, initializing [10218.797339] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10218.802456] Lustre: 318798:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10218.877207] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10224.196326] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [10225.259648] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10225.275719] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10225.395281] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10233.105579] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10237.661689] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10237.859212] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10242.066794] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10242.243423] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10246.119977] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10246.120561] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10246.123262] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10246.130709] LustreError: Skipped 1 previous similar message [10246.157206] Lustre: Skipped 1 previous similar message [10251.242139] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10251.249986] Lustre: Skipped 1 previous similar message [10256.365852] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10256.373231] Lustre: Skipped 1 previous similar message [10257.889290] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10258.172171] Lustre: server umount lustre-OST0000 complete [10263.012924] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10263.028342] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10263.039828] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10263.047643] Lustre: Skipped 1 previous similar message [10266.598172] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10268.369541] Lustre: server umount lustre-MDT0000 complete [10271.852332] LustreError: 316821:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787570537 with bad export cookie 13911572125103137848 [10271.853107] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10271.869097] LustreError: 316821:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10272.310589] Lustre: server umount lustre-MDT0001 complete [10276.386653] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10278.058790] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 07:22:22 (1787570542) [10284.467885] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10284.885921] LustreError: 321111:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10284.907724] LustreError: 321111:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10284.961910] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10289.873970] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [10290.147848] LustreError: 321112:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10295.274171] LustreError: 321111:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10297.855870] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10298.256429] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10302.440529] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [10308.875388] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10314.129851] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10320.469746] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10320.826257] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10327.245117] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [10335.155053] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10342.579501] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10342.769464] Lustre: lustre-OST0001: new disk, initializing [10342.774654] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10342.784185] Lustre: 323949:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10342.896148] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10350.180643] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10350.190964] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10350.261593] LustreError: 324280:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10350.434956] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [10351.335413] LustreError: 324280:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10351.346589] LustreError: 324280:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10352.357314] LustreError: 324289:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10354.474419] LustreError: 324289:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10354.492166] LustreError: 324289:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 4 previous similar messages [10358.564632] LustreError: 324280:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10358.575517] LustreError: 324280:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 6 previous similar messages [10358.984843] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10359.185645] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10364.455806] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff95adc5f02000.ost_server_uuid 50 [10365.802739] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff95adc5f02000.ost_server_uuid in FULL state after 0 sec [10366.823155] LustreError: 324289:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10366.832632] LustreError: 324289:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 16 previous similar messages [10371.216378] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff95adc5f02000.ost_server_uuid 50 [10372.600851] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff95adc5f02000.ost_server_uuid in FULL state after 0 sec [10374.131983] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10389.475598] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10389.477436] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10389.489828] LustreError: Skipped 1 previous similar message [10389.492164] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10389.492170] Lustre: Skipped 2 previous similar messages [10389.520882] Lustre: Skipped 1 previous similar message [10392.745700] Lustre: server umount lustre-OST0001 complete [10396.192854] LustreError: 322933:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.206.59@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10399.715647] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10399.724963] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10401.777652] LustreError: 322932:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10401.792597] LustreError: 322932:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10404.290807] Lustre: server umount lustre-OST0000 complete [10410.978164] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10410.989822] LustreError: Skipped 1 previous similar message [10410.996399] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10410.999354] Lustre: Skipped 7 previous similar messages [10413.851964] Lustre: server umount lustre-MDT0000 complete [10417.082898] LustreError: 321091:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787570682 with bad export cookie 13911572125103138730 [10417.090925] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10417.093269] LustreError: 321091:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10417.491482] Lustre: server umount lustre-MDT0001 complete [10426.851482] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing unload_modules_local [10429.314857] Key type lgssc unregistered [10429.575605] LNet: 326634:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10429.579621] LNetError: 326634:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10429.595377] LNet: Removed LNI 192.168.206.159@tcp [10430.300938] Key type .llcrypt unregistered [10430.304652] Key type ._llcrypt unregistered [10451.051407] Key type ._llcrypt registered [10451.056598] Key type .llcrypt registered [10451.164968] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [10452.046920] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10452.115677] alg: No test for adler32 (adler32-zlib) [10453.235777] Lustre: Lustre: Build Version: 2.17.57_82_g4029ffa [10453.416849] LNet: Added LNI 192.168.206.159@tcp [8/256/0/180] [10455.072282] Key type lgssc registered [10456.140706] Lustre: Echo OBD driver; http://www.lustre.org/ [10463.493603] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10463.520870] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10464.888804] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10468.205362] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [10474.116239] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10475.491143] LustreError: 328035:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10475.501465] LustreError: 328035:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10480.008411] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10480.291282] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10484.047233] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [10492.722487] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10493.035521] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10499.386202] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [10505.329309] LustreError: 329719:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10506.816150] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10510.455367] LustreError: 329717:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10510.466960] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10510.494452] LustreError: 329717:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10512.899134] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10513.114702] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10518.525932] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10518.674130] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [10527.029585] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10533.861689] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10534.042679] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10636.272929] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10636.281650] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10636.294936] LustreError: Skipped 1 previous similar message [10636.295824] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10636.318431] Lustre: Skipped 1 previous similar message [10638.749895] Lustre: server umount lustre-OST0001 complete [10641.385942] LustreError: 329719:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10641.403199] LustreError: 329719:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10646.501864] LustreError: 329717:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10646.507886] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10646.513117] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10646.536909] LustreError: 329717:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10646.558302] Lustre: Skipped 2 previous similar messages [10650.773278] Lustre: server umount lustre-OST0000 complete [10654.690438] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10654.702594] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10654.723505] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10654.735370] Lustre: Skipped 1 previous similar message [10656.737689] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10656.738357] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10656.777823] Lustre: Skipped 1 previous similar message [10660.926140] Lustre: server umount lustre-MDT0000 complete [10661.865554] LustreError: 328035:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10661.893765] LustreError: 328035:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10665.063132] LustreError: 328016:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787570930 with bad export cookie 10900878321724512788 [10665.064916] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10665.080286] LustreError: 328016:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10665.635226] Lustre: server umount lustre-MDT0001 complete [10677.477868] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing unload_modules_local [10680.365578] Key type lgssc unregistered [10680.767444] LNet: 332981:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10680.776499] LNetError: 332981:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10680.797394] LNet: Removed LNI 192.168.206.159@tcp [10681.894845] Key type .llcrypt unregistered [10681.896579] Key type ._llcrypt unregistered [10727.930434] Key type ._llcrypt registered [10727.932889] Key type .llcrypt registered [10728.022695] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [10729.192996] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10729.217702] alg: No test for adler32 (adler32-zlib) [10730.306414] Lustre: Lustre: Build Version: 2.17.57_82_g4029ffa [10730.513476] LNet: Added LNI 192.168.206.159@tcp [8/256/0/180] [10732.168237] Key type lgssc registered [10733.336334] Lustre: Echo OBD driver; http://www.lustre.org/ [10741.781665] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10741.809462] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10743.270687] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10747.249170] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [10753.507091] LustreError: 335270:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10753.520283] LustreError: 335270:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10754.572677] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10754.943098] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10758.955680] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [10762.258689] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [10769.689601] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10769.987492] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10776.755857] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [10780.259738] LustreError: 336762:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10785.571992] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10785.725209] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10787.796083] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [10787.798529] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [10792.955881] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [10802.607288] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10810.095572] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [10821.602583] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10821.610683] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10821.631201] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10823.651775] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10823.662686] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10827.759870] Lustre: server umount lustre-MDT0000 complete [10828.793417] LustreError: 335271:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10828.831645] LustreError: 335271:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10831.583124] LustreError: 335250:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787571096 with bad export cookie 3949067922813435408 [10831.584286] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10831.604877] LustreError: 335250:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10832.056735] Lustre: server umount lustre-MDT0001 complete [10846.575581] Lustre: server umount lustre-OST0000 complete [10860.673529] Lustre: server umount lustre-OST0001 complete [10876.044929] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [10885.868024] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10886.244837] LustreError: 340537:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10886.317328] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10890.423238] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [10891.747240] LustreError: 340538:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10898.659192] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10899.175292] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10903.214304] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [10906.190879] Lustre: 341677:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10912.211877] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10918.341638] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [10924.842401] LustreError: 342032:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10924.882482] LustreError: 342032:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10926.134279] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10926.378846] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10926.384952] Lustre: Skipped 1 previous similar message [10931.520169] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [10931.521096] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [10932.941117] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [10940.190355] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10943.885815] Lustre: 343550:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [10950.117069] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10950.129737] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10950.159618] Lustre: Skipped 2 previous similar messages [10950.168350] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10950.177375] Lustre: Skipped 3 previous similar messages [10952.171254] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10952.175532] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10952.212385] Lustre: Skipped 1 previous similar message [10954.331730] Lustre: server umount lustre-MDT0000 complete [10957.287517] LustreError: 341863:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10957.297952] LustreError: 341863:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10957.587188] LustreError: 340517:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787571222 with bad export cookie 3949067922813437102 [10957.595367] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10957.598746] LustreError: 340517:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10957.978523] Lustre: server umount lustre-MDT0001 complete [10971.824169] Lustre: server umount lustre-OST0000 complete [10987.302456] Lustre: server umount lustre-OST0001 complete [10991.518177] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [10993.062847] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [10994.778601] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 07:34:19 (1787571259) [11018.299626] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [11027.157687] loop0: detected capacity change from 0 to 1099511627776 [11027.793516] SGI XFS with ACLs, security attributes, quota, no debug enabled [11027.798843] *** VALIDATE xfs *** [11027.809634] XFS (loop0): Mounting V5 Filesystem [11027.828276] XFS (loop0): Ending clean mount [11028.966105] loop1: detected capacity change from 0 to 18691697672192 [11132.729621] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_hostid [11141.515184] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [11175.663629] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11175.847757] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11175.863914] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11175.986939] Lustre: lustre-MDT0000: new disk, initializing [11176.081159] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11176.104741] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11181.858957] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [11193.709742] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11193.833492] Lustre: 348684:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11193.910985] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11193.916413] Lustre: Skipped 1 previous similar message [11193.993751] Lustre: lustre-MDT0001: new disk, initializing [11194.115866] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11194.138963] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11198.140869] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [11206.008267] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11210.812723] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11216.936679] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11217.137170] Lustre: lustre-OST0000: new disk, initializing [11217.140744] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11217.145309] Lustre: 350654:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11219.036693] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11219.045356] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11219.136514] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11222.782741] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [11230.377746] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11234.509986] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11234.713849] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11238.890681] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11239.108094] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11245.032400] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11245.032551] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11245.034603] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11245.034610] Lustre: Skipped 3 previous similar messages [11245.049437] LustreError: Skipped 1 previous similar message [11245.068529] Lustre: Skipped 2 previous similar messages [11246.847363] Lustre: server umount lustre-OST0000 complete [11255.269212] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11255.293168] Lustre: Skipped 4 previous similar messages [11256.350454] Lustre: server umount lustre-MDT0000 complete [11259.504289] LustreError: 348676:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787571524 with bad export cookie 3949067922813438656 [11259.509577] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11259.515826] LustreError: 348676:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11259.818263] Lustre: server umount lustre-MDT0001 complete [11266.591228] XFS (loop0): Unmounting Filesystem [11270.083678] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 07:38:54 (1787571534) [11275.474710] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11275.819249] LustreError: 353207:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11275.931833] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11275.936643] Lustre: Skipped 2 previous similar messages [11279.489583] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [11286.018260] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11289.307444] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [11294.446314] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11298.031455] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11302.573980] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11307.531688] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [11312.911468] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11331.553550] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11331.563292] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11331.572337] Lustre: Skipped 2 previous similar messages [11331.579189] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11337.833219] Lustre: server umount lustre-OST0000 complete [11342.818673] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11342.823134] LustreError: Skipped 1 previous similar message [11347.687640] Lustre: server umount lustre-MDT0000 complete [11347.940631] LustreError: 353202:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11347.966558] LustreError: 353202:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [11350.552649] LustreError: 353189:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787571615 with bad export cookie 3949067922813439545 [11350.564234] LustreError: 353189:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11350.564375] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11350.881214] Lustre: server umount lustre-MDT0001 complete [11359.856195] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing unload_modules_local [11362.169332] Key type lgssc unregistered [11362.428674] LNet: 356861:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11362.441536] LNetError: 356861:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11362.455098] LNet: Removed LNI 192.168.206.159@tcp [11363.140134] Key type .llcrypt unregistered [11363.144974] Key type ._llcrypt unregistered [11384.404836] Key type ._llcrypt registered [11384.407420] Key type .llcrypt registered [11384.494803] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [11385.288434] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11385.315526] alg: No test for adler32 (adler32-zlib) [11386.277649] Lustre: Lustre: Build Version: 2.17.57_82_g4029ffa [11386.450198] LNet: Added LNI 192.168.206.159@tcp [8/256/0/180] [11388.096149] Key type lgssc registered [11389.127616] Lustre: Echo OBD driver; http://www.lustre.org/ [11395.472496] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11395.498643] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11396.940356] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11400.750824] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [11407.331983] LustreError: 358406:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11407.355987] LustreError: 358406:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11407.646072] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11407.910821] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11411.637986] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [11417.866437] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11422.590413] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11428.440586] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11428.656373] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11433.741777] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [11440.037687] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11451.960486] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.59@tcp (stopping) [11452.387755] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11452.410469] Lustre: Skipped 1 previous similar message [11457.051215] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.59@tcp (stopping) [11457.061069] Lustre: Skipped 2 previous similar messages [11457.407378] Lustre: server umount lustre-MDT0001 complete [11457.508105] LustreError: 358406:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11457.524352] LustreError: 358406:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11462.168890] LustreError: 358400:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.59@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11466.581782] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11466.699241] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11466.706795] Lustre: Found index 1 for lustre-MDT0001, updating log [11466.748313] Lustre: 358396:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11467.126354] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11471.836384] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [11472.366663] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11473.980342] LustreError: 361942:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11473.993224] LustreError: 361942:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11474.004526] LustreError: 358398:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11474.129551] LustreError: 361956:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.159@tcp: cfg command failed: rc = -17 [11474.136487] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11474.136487] [11482.600036] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11482.616785] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11482.622833] Lustre: Skipped 1 previous similar message [11487.738457] LustreError: 358406:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11487.774412] LustreError: 358406:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11488.231888] Lustre: server umount lustre-MDT0001 complete [11492.843671] LustreError: 358406:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11492.855732] LustreError: 358406:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11499.805479] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11499.959485] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11499.965426] Lustre: Found index 1 for lustre-MDT0001, updating log [11500.009682] Lustre: 358393:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11500.333296] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11504.317301] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [11505.653036] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11506.171890] LustreError: 363045:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11506.178585] LustreError: 363045:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11506.185543] LustreError: 358398:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11506.191661] LustreError: 358398:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11506.253684] LustreError: 363050:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [11506.258292] LustreError: 363050:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [11506.262264] LustreError: 363050:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.159@tcp: cfg command failed: rc = -17 [11506.268700] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.206.159@tcp [11506.268700] [11810.343248] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.59@tcp (stopping) [11810.349121] Lustre: Skipped 1 previous similar message [11812.838632] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11812.845839] Lustre: Skipped 1 previous similar message [11812.859366] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11815.453239] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.59@tcp (stopping) [11815.472668] Lustre: Skipped 2 previous similar messages [11815.662944] Lustre: server umount lustre-MDT0001 complete [11817.954618] LustreError: 358402:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11817.985233] LustreError: 358402:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [11826.159635] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11826.264770] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11826.274657] Lustre: Found index 1 for lustre-MDT0001, updating log [11826.349891] Lustre: 358395:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11826.696909] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [11831.798163] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11831.810134] LustreError: 371945:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11831.817899] LustreError: 371945:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11831.833516] LustreError: 358398:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11831.845446] LustreError: 358398:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11831.956551] LustreError: 371950:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [11832.002063] LustreError: 371950:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.159@tcp: cfg command failed: rc = -114 [11832.034144] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [11832.034144] [11832.847480] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [12154.343289] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12154.351086] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12154.363695] Lustre: Skipped 3 previous similar messages [12154.381205] Lustre: Skipped 2 previous similar messages [12160.047460] Lustre: server umount lustre-MDT0000 complete [12163.889965] LustreError: 361590:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787572429 with bad export cookie 7685905587155636995 [12163.893231] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12163.919762] LustreError: 361590:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12164.577129] LustreError: 364705:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12164.577567] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12164.616509] LustreError: 364705:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [12173.741996] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12173.752717] Lustre: Skipped 5 previous similar messages [12178.400117] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12178.640086] Lustre: server umount lustre-MDT0001 complete [12182.499720] Lustre: server umount lustre-OST0000 complete [12192.663092] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_hostid [12203.037314] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [12241.146874] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 07:55:05 (1787572505) [12247.676143] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12247.972293] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12248.009208] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12248.104444] Lustre: lustre-MDT0000: new disk, initializing [12248.188984] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12248.237985] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12252.999267] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [12264.846574] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12265.036660] Lustre: 382935:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12265.142175] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12265.151203] Lustre: Skipped 1 previous similar message [12265.245334] Lustre: lustre-MDT0001: new disk, initializing [12265.418757] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12265.470325] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12265.483416] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12269.937616] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [12279.251141] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12285.325705] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12292.637136] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12293.019126] Lustre: lustre-OST0000: new disk, initializing [12293.024354] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12293.035953] Lustre: 384907:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12293.172830] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12299.413994] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_default_debug -1 all [12300.875871] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12300.886220] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12301.015705] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12308.962591] Lustre: DEBUG MARKER: oleg659-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12316.642832] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12316.647862] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12316.648825] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12316.654231] Lustre: Skipped 1 previous similar message [12321.945685] Lustre: server umount lustre-MDT0000 complete [12325.690524] LustreError: 382928:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787572591 with bad export cookie 7685905587155638948 [12325.695404] LustreError: MGC192.168.206.159@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12325.700899] LustreError: 382928:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12325.710056] LustreError: 383742:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [12326.155319] Lustre: server umount lustre-MDT0001 complete [12340.597973] Lustre: server umount lustre-OST0000 complete [12370.382650] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing set_hostid [12381.566618] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing load_modules_local [12411.843360] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12130 sec ========= 07:57:56 (1787572676) [12413.334941] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 07:57:57 (1787572677) === [12416.521626] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 07:58:01 (1787572681) === [12461.532662] Lustre: DEBUG MARKER: oleg659-server.virtnet: executing unload_modules_local [12464.314273] Key type lgssc unregistered [12464.571980] LNet: 391276:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12464.586047] LNetError: 391276:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12464.625212] LNet: Removed LNI 192.168.206.159@tcp [12465.579067] Key type .llcrypt unregistered [12465.583949] Key type ._llcrypt unregistered