[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 554035333 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a2000-0x1465ccfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524588K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001013] APIC: Switch to symmetric I/O mode setup [ 0.002340] x2apic enabled [ 0.003010] Switched APIC routing to physical x2apic. [ 0.004019] kvm-guest: setup PV IPIs [ 0.007000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007028] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009010] pid_max: default: 32768 minimum: 301 [ 0.010145] LSM: Security Framework initializing [ 0.011060] Yama: becoming mindful. [ 0.013040] SELinux: Initializing. [ 0.014098] *** VALIDATE selinux *** [ 0.022744] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027375] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028173] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030031] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.032056] *** VALIDATE tmpfs *** [ 0.033511] *** VALIDATE proc *** [ 0.035278] *** VALIDATE cgroup *** [ 0.036012] *** VALIDATE cgroup2 *** [ 0.038080] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.039162] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.040011] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.041038] Spectre V2 : User space: Vulnerable [ 0.042011] Speculative Store Bypass: Vulnerable [ 0.045160] debug: unmapping init [mem 0xffffffffb8e59000-0xffffffffb8e60fff] [ 0.048182] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.049757] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.050024] ... version: 2 [ 0.051015] ... bit width: 48 [ 0.052015] ... generic registers: 4 [ 0.053013] ... value mask: 0000ffffffffffff [ 0.054018] ... max period: 00007fffffffffff [ 0.055018] ... fixed-purpose events: 3 [ 0.056015] ... event mask: 000000070000000f [ 0.058248] rcu: Hierarchical SRCU implementation. [ 0.060505] smp: Bringing up secondary CPUs ... [ 0.061649] x86: Booting SMP configuration: [ 0.062036] .... node #0, CPUs: #1 #2 #3 [ 0.065511] smp: Brought up 1 node, 4 CPUs [ 0.066901] smpboot: Max logical packages: 1 [ 0.067018] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.229927] node 0 deferred pages initialised in 160ms [ 0.232329] devtmpfs: initialized [ 0.233250] x86/mm: Memory block size: 128MB [ 0.236174] gcov: version magic: 0x41383552 [ 0.240302] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.241078] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.242303] pinctrl core: initialized pinctrl subsystem [ 0.243205] [ 0.243741] ************************************************************* [ 0.244018] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.245029] ** ** [ 0.246017] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.247017] ** ** [ 0.248020] ** This means that this kernel is built to expose internal ** [ 0.249016] ** IOMMU data structures, which may compromise security on ** [ 0.250022] ** your system. ** [ 0.251016] ** ** [ 0.252017] ** If you see this message and you are not debugging the ** [ 0.253017] ** kernel, report this immediately to your vendor! ** [ 0.254019] ** ** [ 0.255019] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.256017] ************************************************************* [ 0.257696] NET: Registered protocol family 16 [ 0.258431] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.259085] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.260091] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.262046] cpuidle: using governor menu [ 0.263814] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.266624] PCI: Using configuration type 1 for base access [ 0.269155] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.280149] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.281035] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.283072] cryptd: max_cpu_qlen set to 1000 [ 0.286288] ACPI: Added _OSI(Module Device) [ 0.287021] ACPI: Added _OSI(Processor Device) [ 0.288016] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.290014] ACPI: Added _OSI(Processor Aggregator Device) [ 0.294479] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.301595] ACPI: Interpreter enabled [ 0.303085] ACPI: PM: (supports S0 S3 S4 S5) [ 0.304013] ACPI: Using IOAPIC for interrupt routing [ 0.306137] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.310526] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.321950] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.324059] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.327028] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.331117] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.336667] acpiphp: Slot [2] registered [ 0.338177] acpiphp: Slot [5] registered [ 0.339213] acpiphp: Slot [6] registered [ 0.341163] acpiphp: Slot [7] registered [ 0.343258] acpiphp: Slot [8] registered [ 0.344138] acpiphp: Slot [9] registered [ 0.346144] acpiphp: Slot [10] registered [ 0.347211] acpiphp: Slot [3] registered [ 0.349147] acpiphp: Slot [4] registered [ 0.351147] acpiphp: Slot [11] registered [ 0.352147] acpiphp: Slot [12] registered [ 0.354123] acpiphp: Slot [13] registered [ 0.355123] acpiphp: Slot [14] registered [ 0.356119] acpiphp: Slot [15] registered [ 0.358089] acpiphp: Slot [16] registered [ 0.359103] acpiphp: Slot [17] registered [ 0.360100] acpiphp: Slot [18] registered [ 0.361085] acpiphp: Slot [19] registered [ 0.362116] acpiphp: Slot [20] registered [ 0.363089] acpiphp: Slot [21] registered [ 0.364119] acpiphp: Slot [22] registered [ 0.366080] acpiphp: Slot [23] registered [ 0.367096] acpiphp: Slot [24] registered [ 0.368084] acpiphp: Slot [25] registered [ 0.370129] acpiphp: Slot [26] registered [ 0.371128] acpiphp: Slot [27] registered [ 0.373166] acpiphp: Slot [28] registered [ 0.374076] acpiphp: Slot [29] registered [ 0.375082] acpiphp: Slot [30] registered [ 0.377137] acpiphp: Slot [31] registered [ 0.379106] PCI host bridge to bus 0000:00 [ 0.380034] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.382028] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.384028] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.386039] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.389042] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.391029] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.393226] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.396090] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.399398] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.409020] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.415071] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.418021] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.419019] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.422025] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.426013] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.427661] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.429033] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.433030] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.438021] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.452022] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.456027] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.465057] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.476026] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.486023] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.509022] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.530189] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.542023] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.555024] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.580026] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.591840] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.601026] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.614024] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.632025] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.642495] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.651022] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.658020] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.672027] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.682939] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.693025] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.706023] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.728027] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.745844] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.753034] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.764026] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.782468] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.799052] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.801530] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.804436] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.807853] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.810295] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.815167] iommu: Default domain type: Passthrough [ 0.816550] SCSI subsystem initialized [ 0.818168] ACPI: bus type USB registered [ 0.820152] usbcore: registered new interface driver usbfs [ 0.821100] usbcore: registered new interface driver hub [ 0.824117] usbcore: registered new device driver usb [ 0.825220] pps_core: LinuxPPS API ver. 1 registered [ 0.827016] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.830085] PTP clock support registered [ 0.832186] EDAC MC: Ver: 3.0.0 [ 0.835215] PCI: Using ACPI for IRQ routing [ 0.838102] NetLabel: Initializing [ 0.839012] NetLabel: domain hash size = 128 [ 0.840012] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.842114] NetLabel: unlabeled traffic allowed by default [ 0.844136] vgaarb: loaded [ 0.846345] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.848023] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.855707] clocksource: Switched to clocksource kvm-clock [ 0.976980] VFS: Disk quotas dquot_6.6.0 [ 0.978732] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.981526] *** VALIDATE ramfs *** [ 0.982925] *** VALIDATE hugetlbfs *** [ 0.984723] pnp: PnP ACPI init [ 0.987445] pnp: PnP ACPI: found 6 devices [ 1.006176] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.009702] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.012078] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.014555] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.017216] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.019951] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.022752] NET: Registered protocol family 2 [ 1.025369] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.030796] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.034445] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.039660] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.042942] TCP: Hash tables configured (established 65536 bind 65536) [ 1.046064] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.048782] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.051648] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.054681] NET: Registered protocol family 1 [ 1.057471] RPC: Registered named UNIX socket transport module. [ 1.063667] RPC: Registered udp transport module. [ 1.065737] RPC: Registered tcp transport module. [ 1.067613] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.070328] NET: Registered protocol family 44 [ 1.072196] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.074579] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.076841] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.079452] PCI: CLS 0 bytes, default 64 [ 1.081242] Unpacking initramfs... [ 2.512225] debug: unmapping init [mem 0xffff8b7fbcc54000-0xffff8b7fbffbffff] [ 2.514802] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.516425] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.518355] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.003344] Initialise system trusted keyrings [ 3.005041] Key type blacklist registered [ 3.007451] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.017813] zbud: loaded [ 3.020852] *** VALIDATE nfs *** [ 3.022137] *** VALIDATE nfs4 *** [ 3.023775] pstore: using deflate compression [ 3.027398] Platform Keyring initialized [ 3.148247] NET: Registered protocol family 38 [ 3.149408] Key type asymmetric registered [ 3.150521] Asymmetric key parser 'x509' registered [ 3.151902] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.154112] io scheduler mq-deadline registered [ 3.155240] io scheduler kyber registered [ 3.156399] io scheduler bfq registered [ 3.158278] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.161377] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.164270] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.167545] ACPI: Power Button [PWRF] [ 3.173109] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.180165] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.193232] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.199897] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.213035] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.238795] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.266193] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.270789] Non-volatile memory driver v1.3 [ 3.272712] Linux agpgart interface v0.103 [ 3.302586] virtio_blk virtio1: [vda] 150008 512-byte logical blocks (76.8 MB/73.2 MiB) [ 3.304897] vda: detected capacity change from 0 to 76804096 [ 3.318760] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.320978] vdb: detected capacity change from 0 to 1073741824 [ 3.337209] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.339349] vdc: detected capacity change from 0 to 2621440000 [ 3.353266] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.355627] vdd: detected capacity change from 0 to 2621440000 [ 3.368307] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.371628] vde: detected capacity change from 0 to 4294967296 [ 3.389240] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.392765] vdf: detected capacity change from 0 to 4294967296 [ 3.400630] libphy: Fixed MDIO Bus: probed [ 3.408389] usbcore: registered new interface driver usbserial_generic [ 3.410853] usbserial: USB Serial support registered for generic [ 3.413179] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.417875] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.419288] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.421118] mousedev: PS/2 mouse device common for all mice [ 3.423145] rtc_cmos 00:05: RTC can wake from S4 [ 3.425859] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.427189] rtc_cmos 00:05: registered as rtc0 [ 3.431774] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.435032] intel_pstate: CPU model not supported [ 3.437395] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.440679] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.441068] hid: raw HID events driver (C) Jiri Kosina [ 3.445908] usbcore: registered new interface driver usbhid [ 3.448133] usbhid: USB HID core driver [ 3.449789] drop_monitor: Initializing network drop monitor service [ 3.452399] Initializing XFRM netlink socket [ 3.454660] NET: Registered protocol family 10 [ 3.458453] Segment Routing with IPv6 [ 3.460097] NET: Registered protocol family 17 [ 3.462824] mpls_gso: MPLS GSO support [ 3.468566] RAS: Correctable Errors collector initialized. [ 3.469989] AVX version of gcm_enc/dec engaged. [ 3.471862] AES CTR mode by8 optimization enabled [ 3.552761] sched_clock: Marking stable (3552725624, 0)->(4480256198, -927530574) [ 3.556750] registered taskstats version 1 [ 3.558859] Loading compiled-in X.509 certificates [ 3.560908] zswap: loaded using pool lzo/zbud [ 3.586965] Key type big_key registered [ 3.601646] Key type encrypted registered [ 3.603452] ima: No TPM chip found, activating TPM-bypass! [ 3.605712] ima: Allocated hash algorithm: sha1 [ 3.607438] ima: No architecture policies found [ 3.609591] evm: Initialising EVM extended attributes: [ 3.611675] evm: security.selinux [ 3.612884] evm: security.ima [ 3.613678] evm: security.capability [ 3.614617] evm: HMAC attrs: 0x1 [ 3.616692] rtc_cmos 00:05: setting system clock to 2026-09-15 04:51:47 UTC (1789447907) [ 3.621278] debug: unmapping init [mem 0xffffffffb9e03000-0xffffffffb9ffffff] [ 3.623215] debug: unmapping init [mem 0xffffffffb8b82000-0xffffffffb8e58fff] [ 3.631108] Write protecting the kernel read-only data: 28672k [ 3.633979] debug: unmapping init [mem 0xffffffffb7203000-0xffffffffb73fffff] [ 3.635922] debug: unmapping init [mem 0xffffffffb7b14000-0xffffffffb7bfffff] [ 3.671542] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.678390] systemd[1]: Detected virtualization kvm. [ 3.680742] systemd[1]: Detected architecture x86-64. [ 3.682981] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.711516] systemd[1]: No hostname configured. [ 3.713373] systemd[1]: Set hostname to . [ 3.715900] random: systemd: uninitialized urandom read (16 bytes read) [ 3.718772] systemd[1]: Initializing machine ID from random generator. [ 3.775168] random: ln: uninitialized urandom read (6 bytes read) [ 3.860400] random: systemd: uninitialized urandom read (16 bytes read) [ 3.862520] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 3.866975] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 3.871622] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Reached target Timers. [ OK ] Reached target Initrd Root Device. Starting Create Volatile Files and Directories... [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on udev Control Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Sockets. Starting Apply Kernel Variables... [ OK ] Reached target Swap. [ OK ] Reached target Slices. Starting Setup Virtual Console... [ OK ] Reached target Paths. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Memstrack Anylazing Service. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.484253] device-mapper: uevent: version 1.0.3 [ 4.486781] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.182691] virtio_net virtio0 ens2: renamed from eth0 [ 5.194852] random: fast init done [ 5.268207] scsi host0: ata_piix [ 5.299867] scsi host1: ata_piix [ 5.301705] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.304117] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.699746] dracut-initqueue[585]: RTNETLINK answers: File exists [ 10.119561] random: crng init done [ 10.121029] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.517931] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.636199] printk: systemd: 24 output lines suppressed due to ratelimiting [ 11.867041] SELinux: Disabled at runtime. [ 11.933839] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.944186] systemd[1]: Detected virtualization kvm. [ 11.946229] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.420271] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.424390] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.429642] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.433837] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.437162] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.443818] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.450329] systemd[1]: Mounting POSIX Message Queue File System... Mounting POSIX Message Queue File System... [ OK ] Stopped target Switch Root. [ OK ] Started Forward Password Requests to Wall Directory Watch. Starting Create list of required st…ce nodes for the current kernel... Mounting Kernel Debug File System... [ OK ] Stopped target Initrd Root File System. Starting Remount Root and Kernel File Systems... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Listening on Process Core Dump Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. Mounting Huge Pages File System... [ OK ] Created slice system-serial\x2dgetty.slice. Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Listening on udev Kernel Socket. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Created slice system-getty.slice. [ 12.556339] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Starting Apply Kernel Variables... [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Stopped target Initrd File Systems. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Reached target rpc_pipefs.target. [ OK ] Reached target Paths. [ OK ] Started Journal Service. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Kernel Debug File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Huge Pages File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Apply Kernel Variables. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Mounted /mnt. [ 12.812612] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Coldplug all Devices. [ OK ] Started udev Kernel Device Manager. [ 13.073676] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.093767] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.220619] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.234967] EDAC sbridge: Ver: 1.1.2 [ 14.639894] Key type dns_resolver registered [ 14.932105] NFS: Registering the id_resolver key type [ 14.934209] Key type id_resolver registered [ 14.935785] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Load/Save Random Seed... Starting Create Volatile Files and Directories... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Login Service... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target sshd-keygen.target. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... Starting Hostname Service... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting System Logging Service... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg124-server login: [ 27.553036] hrtimer: interrupt took 2816490 ns [ 74.324938] libcfs: loading out-of-tree module taints kernel. [ 74.371929] Key type ._llcrypt registered [ 74.375329] Key type .llcrypt registered [ 74.481429] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [ 96.975492] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 98.993164] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 99.007494] alg: No test for adler32 (adler32-zlib) [ 100.593326] Lustre: Lustre: Build Version: 2.17.58_40_gc6c2479 [ 101.760506] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 103.568239] Key type lgssc registered [ 105.585676] Lustre: Echo OBD driver; http://www.lustre.org/ [ 126.336730] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 174.791809] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 188.706363] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 188.760271] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 190.060349] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 190.094915] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 190.223782] Lustre: lustre-MDT0000: new disk, initializing [ 190.304368] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 190.320877] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 194.890380] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 208.145970] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 208.260777] Lustre: 6508:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 208.289441] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 208.307849] Lustre: Skipped 1 previous similar message [ 208.397355] Lustre: lustre-MDT0001: new disk, initializing [ 208.466341] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 208.506230] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 208.534337] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 213.320543] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 219.054802] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 228.867264] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 229.169990] Lustre: lustre-OST0000: new disk, initializing [ 229.177837] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 229.192197] Lustre: 8446:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 229.283052] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 230.741082] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 230.749104] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 230.806630] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 236.542578] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 251.592550] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 251.752320] Lustre: lustre-OST0001: new disk, initializing [ 251.755727] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 251.761863] Lustre: 9519:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 251.837138] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 258.064196] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 258.072078] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 258.149101] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 258.241070] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 271.223329] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 277.469983] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 283.600464] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing check_logdir /tmp/testlogs/ [ 288.431322] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing yml_node [ 293.503088] Lustre: DEBUG MARKER: Client: 2.17.58.40 [ 296.409514] Lustre: DEBUG MARKER: MDS: 2.17.58.40 [ 299.571805] Lustre: DEBUG MARKER: OSS: 2.17.58.40 [ 301.443683] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Tue Sep 15 00:56:43 EDT 2026 [ 319.985652] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 322.365821] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 339.938594] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 339.941764] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 339.960021] Lustre: Skipped 3 previous similar messages [ 339.973148] Lustre: Skipped 3 previous similar messages [ 345.059202] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 345.061792] Lustre: Skipped 3 previous similar messages [ 350.178150] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 353.248227] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 353.486428] Lustre: server umount lustre-MDT0000 complete [ 357.374709] LustreError: 8444:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789448261 with bad export cookie 13178953723771976737 [ 357.381181] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 357.393256] LustreError: 8444:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 360.419516] LustreError: 6515:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 360.420678] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 360.438209] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 360.438218] Lustre: Skipped 3 previous similar messages [ 360.447403] LustreError: 6515:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 6 previous similar messages [ 360.464722] Lustre: Skipped 1 previous similar message [ 362.466193] LustreError: 6514:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 362.505217] LustreError: 6514:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 363.934752] Lustre: server umount lustre-MDT0001 complete [ 368.587584] Lustre: server umount lustre-OST0000 complete [ 372.506434] Lustre: server umount lustre-OST0001 complete [ 378.843334] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [ 389.909406] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 425.758679] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 425.975310] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 426.012380] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 426.102689] Lustre: lustre-MDT0000: new disk, initializing [ 426.246071] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 426.267914] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 430.602432] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 442.750635] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 442.886996] Lustre: 15074:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 442.925683] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 442.930132] Lustre: Skipped 1 previous similar message [ 442.991598] Lustre: lustre-MDT0001: new disk, initializing [ 443.068037] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 443.108193] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 443.122198] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 447.138427] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 456.129547] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 461.964918] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 469.011193] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 469.259422] Lustre: lustre-OST0000: new disk, initializing [ 469.262719] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 469.270144] Lustre: 17043:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 469.362338] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 470.611556] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 470.617596] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 470.765221] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 475.487380] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 485.956756] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 490.803292] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 490.992662] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 496.276565] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 496.557216] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 501.750295] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 501.756808] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 501.779579] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 501.786547] Lustre: Skipped 3 previous similar messages [ 505.214672] Lustre: server umount lustre-OST0000 complete [ 509.923958] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 509.933618] LustreError: Skipped 1 previous similar message [ 509.938628] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 509.955680] Lustre: Skipped 1 previous similar message [ 509.965819] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 524.256120] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 524.496463] Lustre: server umount lustre-MDT0000 complete [ 526.318554] LustreError: 15085:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 529.130316] LustreError: 15064:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789448433 with bad export cookie 13178953723771978270 [ 529.133075] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 529.141805] LustreError: 15064:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 529.698790] Lustre: server umount lustre-MDT0001 complete [ 535.771725] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 01:00:37 (1789448437) [ 542.614346] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 542.988626] LustreError: 19310:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 543.007463] LustreError: 19310:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 543.071067] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 547.760413] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 548.329121] LustreError: 19310:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 548.356134] LustreError: 19310:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 556.713501] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 562.030809] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 569.814618] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 575.398037] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 583.538584] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 591.848058] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 602.036560] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 607.093286] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 616.933694] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 616.935364] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 616.958174] Lustre: Skipped 3 previous similar messages [ 616.975246] Lustre: Skipped 8 previous similar messages [ 620.929197] Lustre: server umount lustre-MDT0000 complete [ 622.057144] LustreError: 20313:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 622.100482] LustreError: 20313:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 625.374265] LustreError: 19292:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789448529 with bad export cookie 13178953723771979152 [ 625.378811] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 625.382246] LustreError: 19292:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 625.958923] Lustre: server umount lustre-MDT0001 complete [ 641.152861] Lustre: server umount lustre-OST0000 complete [ 664.772685] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 674.837897] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 675.307854] LustreError: 23706:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 675.373901] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 675.385302] Lustre: Skipped 2 previous similar messages [ 681.017279] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 690.943329] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 696.375733] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 699.589536] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 706.843908] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 714.006830] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 724.372089] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 724.620832] Lustre: lustre-OST0001: new disk, initializing [ 724.625850] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 724.632976] Lustre: 26022:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 726.271483] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 726.280135] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 726.456800] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 731.491873] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 741.622888] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 745.523179] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 757.221029] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 757.238522] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 757.258205] Lustre: Skipped 2 previous similar messages [ 757.267646] Lustre: Skipped 1 previous similar message [ 763.624700] Lustre: server umount lustre-MDT0000 complete [ 767.332882] LustreError: 23687:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789448671 with bad export cookie 13178953723771979999 [ 767.351227] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 767.458845] LustreError: 23701:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 767.487147] LustreError: 23701:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [ 767.802592] Lustre: server umount lustre-MDT0001 complete [ 774.007925] Lustre: server umount lustre-OST0000 complete [ 778.783369] Lustre: server umount lustre-OST0001 complete [ 796.319634] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 809.450191] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 810.096241] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 810.106208] Lustre: Skipped 3 previous similar messages [ 815.044070] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 824.332452] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 829.427657] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 832.432592] Lustre: 30187:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 839.531569] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 846.423942] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 850.114619] LustreError: 30541:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 850.138119] LustreError: 30541:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 5 previous similar messages [ 855.411857] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 862.902782] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 870.590409] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 874.803613] Lustre: 32064:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 886.244221] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 886.253438] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 886.262127] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 886.262135] Lustre: Skipped 8 previous similar messages [ 886.272612] Lustre: Skipped 5 previous similar messages [ 887.755766] Lustre: server umount lustre-MDT0000 complete [ 891.870672] LustreError: 29019:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789448795 with bad export cookie 13178953723771981294 [ 891.875617] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 891.880298] LustreError: 29019:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 892.483120] Lustre: server umount lustre-MDT0001 complete [ 906.945214] Lustre: server umount lustre-OST0000 complete [ 921.494780] Lustre: server umount lustre-OST0001 complete [ 930.423934] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 01:07:12 (1789448832) [ 959.323157] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 969.947394] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 975.936505] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 980.962379] LustreError: 34869:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 980.979136] LustreError: 34869:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 8 previous similar messages [ 985.981684] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 991.346564] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 994.396563] Lustre: 36019:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1002.329494] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1010.525991] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1019.648199] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1026.854546] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1035.617739] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1039.872485] Lustre: 37897:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1045.991396] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1046.003528] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1046.007764] Lustre: Skipped 3 previous similar messages [ 1046.021825] Lustre: Skipped 4 previous similar messages [ 1054.354854] Lustre: server umount lustre-MDT0000 complete [ 1058.047067] LustreError: 34851:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789448961 with bad export cookie 13178953723771982400 [ 1058.057133] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1058.068756] LustreError: 34851:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1058.666777] Lustre: server umount lustre-MDT0001 complete [ 1073.573581] Lustre: server umount lustre-OST0000 complete [ 1087.767200] Lustre: server umount lustre-OST0001 complete [ 1097.749691] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 01:09:59 (1789448999) [ 1115.716908] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 1126.429262] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1127.017070] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1127.031595] Lustre: Skipped 7 previous similar messages [ 1131.832111] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1141.256433] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1147.064061] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1150.506538] Lustre: 41190:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1159.559504] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1168.754901] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1178.593344] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1185.241945] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1194.015760] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1198.026815] Lustre: 43068:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1199.636565] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1199.644709] Lustre: Skipped 1 previous similar message [ 1203.170137] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1203.175289] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1203.193043] Lustre: Skipped 1 previous similar message [ 1205.418130] Lustre: server umount lustre-MDT0000 complete [ 1210.252057] LustreError: 40021:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789449114 with bad export cookie 13178953723771983506 [ 1210.254481] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1210.266480] LustreError: 40021:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1216.810474] Lustre: server umount lustre-MDT0001 complete [ 1225.026221] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1235.440446] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0xb6e50c9630bd9b0e [ 1235.447852] Lustre: MGC192.168.201.124@tcp: Connection restored to 0@lo (at 0@lo) [ 1239.726841] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1240.933984] LustreError: 43946:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1240.974144] LustreError: 43946:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 29 previous similar messages [ 1246.061415] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 1247.176564] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1251.008441] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1252.852922] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1252.859793] Lustre: Skipped 1 previous similar message [ 1257.783225] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1259.302844] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1264.177384] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1265.778117] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1273.319130] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1276.751182] Lustre: server umount lustre-MDT0000 complete [ 1279.886806] LustreError: 40022:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789449183 with bad export cookie 13178953723771984654 [ 1279.890505] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1279.905271] LustreError: 40022:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1280.125542] Lustre: server umount lustre-MDT0001 complete [ 1293.881770] Lustre: server umount lustre-OST0000 complete [ 1306.531617] Lustre: server umount lustre-OST0001 complete [ 1314.753367] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 01:13:36 (1789449216) [ 1328.904375] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 1338.659341] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1344.128096] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1354.841364] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1359.191886] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1361.973722] Lustre: 48843:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1369.512939] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1376.755804] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1385.384221] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1391.341442] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1399.865528] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1404.872312] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1416.672874] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1416.680242] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1416.690371] Lustre: Skipped 9 previous similar messages [ 1416.693605] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1416.701351] Lustre: Skipped 23 previous similar messages [ 1418.993531] Lustre: server umount lustre-MDT0000 complete [ 1422.450167] LustreError: 47670:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789449326 with bad export cookie 13178953723771985473 [ 1422.458054] LustreError: 47670:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1422.460195] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1423.202731] Lustre: server umount lustre-MDT0001 complete [ 1436.469179] Lustre: server umount lustre-OST0000 complete [ 1450.799531] Lustre: server umount lustre-OST0001 complete [ 1462.351383] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 01:16:04 (1789449364) [ 1465.726708] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1467.431903] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 01:16:09 (1789449369) [ 1478.917873] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [ 1489.835369] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 1515.855980] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1516.090579] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1516.184274] Lustre: lustre-MDT0000: new disk, initializing [ 1516.303778] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1519.591170] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1530.161737] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1537.333286] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1537.809948] Lustre: lustre-OST0000: new disk, initializing [ 1537.813194] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1537.816681] Lustre: Skipped 1 previous similar message [ 1537.820668] Lustre: 54913:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1539.262628] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1539.275546] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1539.318173] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1544.615984] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1554.472144] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1557.096190] Lustre: DEBUG MARKER: create test files [ 1595.497575] Lustre: server umount lustre-OST0000 complete [ 1600.511704] Lustre: server umount lustre-MDT0000 complete [ 1612.922664] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 1615.926520] Key type lgssc unregistered [ 1616.252609] LNet: 56542:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1616.256527] LNetError: 56542:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1616.279151] LNet: Removed LNI 192.168.201.124@tcp [ 1617.393159] Key type .llcrypt unregistered [ 1617.397072] Key type ._llcrypt unregistered [ 1650.857856] Key type ._llcrypt registered [ 1650.860149] Key type .llcrypt registered [ 1650.982978] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 1652.173466] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1652.269483] alg: No test for adler32 (adler32-zlib) [ 1653.557772] Lustre: Lustre: Build Version: 2.17.58_40_gc6c2479 [ 1653.963530] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 1655.688331] Key type lgssc registered [ 1657.141135] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1666.762277] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1666.796118] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1668.307136] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1674.104598] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1682.634418] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1690.467957] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1690.744731] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1693.814566] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1693.886819] Lustre: 59275:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1693.892854] Lustre: 59275:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1693.896805] Lustre: 59275:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1693.901384] Lustre: 59275:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1693.905139] Lustre: 59275:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1693.908924] Lustre: 59275:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1697.609946] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1705.880595] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1711.486995] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1718.222607] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1730.017792] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1730.032792] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1730.049474] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1732.545659] Lustre: server umount lustre-OST0000 complete [ 1736.648628] Lustre: server umount lustre-MDT0000 complete [ 1749.290292] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 1752.156580] Key type lgssc unregistered [ 1752.497170] LNet: 60882:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1752.508227] LNetError: 60882:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1752.533654] LNet: Removed LNI 192.168.201.124@tcp [ 1753.311452] Key type .llcrypt unregistered [ 1753.315777] Key type ._llcrypt unregistered [ 1789.128537] Key type ._llcrypt registered [ 1789.133302] Key type .llcrypt registered [ 1789.240402] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 1790.525360] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1790.750621] alg: No test for adler32 (adler32-zlib) [ 1792.018833] Lustre: Lustre: Build Version: 2.17.58_40_gc6c2479 [ 1792.406944] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 1794.088220] Key type lgssc registered [ 1795.451226] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1804.145733] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1804.157501] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1805.537732] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1810.751811] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1819.273312] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1825.760103] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1826.115524] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1830.273154] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1830.365445] Lustre: 63716:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1830.383518] Lustre: 63716:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1830.392182] Lustre: 63716:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1830.398027] Lustre: 63716:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1830.413366] Lustre: 63716:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1830.419213] Lustre: 63716:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1832.600578] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1840.202516] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1847.220315] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1861.091188] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1861.122916] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1862.999585] Lustre: server umount lustre-OST0000 complete [ 1867.844435] Lustre: server umount lustre-MDT0000 complete [ 1881.167169] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 1884.088392] Key type lgssc unregistered [ 1884.519826] LNet: 65271:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1884.545352] LNetError: 65271:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1884.579465] LNet: Removed LNI 192.168.201.124@tcp [ 1885.624756] Key type .llcrypt unregistered [ 1885.629302] Key type ._llcrypt unregistered [ 1904.748464] Key type ._llcrypt registered [ 1904.756168] Key type .llcrypt registered [ 1904.955430] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [ 1920.980398] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 1922.346229] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1922.370067] alg: No test for adler32 (adler32-zlib) [ 1923.608970] Lustre: Lustre: Build Version: 2.17.58_40_gc6c2479 [ 1923.922463] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 1925.656210] Key type lgssc registered [ 1927.042657] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1967.469931] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1967.488982] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1968.786580] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1968.828727] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1968.888305] Lustre: lustre-MDT0000: new disk, initializing [ 1969.027567] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1969.060307] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1973.893295] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1984.676524] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1984.776724] Lustre: 68426:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1984.831441] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1984.836916] Lustre: Skipped 1 previous similar message [ 1984.901520] Lustre: lustre-MDT0001: new disk, initializing [ 1984.991211] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1985.041583] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1985.055556] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1989.058194] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1999.506204] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2007.682550] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2018.957297] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2019.501895] Lustre: lustre-OST0000: new disk, initializing [ 2019.509463] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2019.533413] Lustre: 70395:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2019.691182] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2025.066902] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2025.082317] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2025.297695] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2026.691294] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 2039.155106] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2044.963306] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2045.266803] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2050.330228] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2050.577341] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2056.166908] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2056.172379] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2056.197895] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2056.216216] Lustre: Skipped 1 previous similar message [ 2059.128269] Lustre: server umount lustre-OST0000 complete [ 2066.401626] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2066.413015] Lustre: Skipped 1 previous similar message [ 2066.425930] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2066.444087] Lustre: Skipped 1 previous similar message [ 2069.892571] Lustre: server umount lustre-MDT0000 complete [ 2074.168829] LustreError: 68417:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789449978 with bad export cookie 13423448056144910819 [ 2074.169120] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2074.184805] LustreError: 68417:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2075.501450] Lustre: server umount lustre-MDT0001 complete [ 2086.497830] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 01:26:28 (1789449988) [ 2099.539951] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 2140.060885] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [ 2149.536576] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 2192.586978] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2193.127267] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2193.171879] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2193.268110] Lustre: lustre-MDT0000: new disk, initializing [ 2193.363256] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2193.373476] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2199.347523] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 2209.420640] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2209.500573] Lustre: 76414:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2209.533211] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2209.536273] Lustre: Skipped 1 previous similar message [ 2209.587277] Lustre: lustre-MDT0001: new disk, initializing [ 2209.659616] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2209.697391] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2209.707818] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2213.727531] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 2222.990463] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2228.126197] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2233.866323] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2234.148660] Lustre: lustre-OST0000: new disk, initializing [ 2234.157727] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2234.164229] Lustre: 78384:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2234.273281] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2239.757479] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 2243.665221] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2243.683544] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2243.765365] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2248.667261] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2252.797935] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2252.979656] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2256.664810] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2256.787661] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2259.425480] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2259.429144] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2259.431802] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2259.448676] Lustre: Skipped 2 previous similar messages [ 2259.460982] Lustre: Skipped 1 previous similar message [ 2264.563289] LustreError: 78366:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2264.578865] LustreError: 78366:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 2264.622587] Lustre: server umount lustre-OST0000 complete [ 2269.669470] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2269.683054] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2269.693041] Lustre: Skipped 1 previous similar message [ 2274.774270] Lustre: server umount lustre-MDT0000 complete [ 2274.793726] LustreError: 76423:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2274.822809] LustreError: 76423:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 2278.136232] LustreError: 76407:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789450182 with bad export cookie 13423448056144911701 [ 2278.152324] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2278.548452] Lustre: server umount lustre-MDT0001 complete [ 2286.980930] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 01:29:49 (1789450189) [ 2293.525622] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2294.240842] LustreError: 80795:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2294.384859] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2298.262423] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 2299.367743] LustreError: 80796:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2304.483820] LustreError: 80795:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2307.374687] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2311.697330] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 2320.453219] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2326.425670] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2332.790548] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2333.233934] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2333.239166] Lustre: Skipped 1 previous similar message [ 2339.902721] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 2347.569200] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2350.873747] Lustre: 83465:0:(genops.c:1773:obd_export_evict_by_uuid()) MGS: evicting d16c2a66-7b33-4a73-abad-0e8167e8dcc2 at adminstrative request [ 2352.678273] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2352.713794] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2362.865951] Lustre: MGS: Client d16c2a66-7b33-4a73-abad-0e8167e8dcc2 (at 0@lo) reconnecting [ 2362.872348] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xba49aae2f76ce0c7 to 0xba49aae2f76ce367 [ 2362.882817] Lustre: MGC192.168.201.124@tcp: Connection restored to 0@lo (at 0@lo) [ 2388.570676] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2388.795907] Lustre: lustre-OST0001: new disk, initializing [ 2388.799432] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2388.806740] Lustre: 83858:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2395.054218] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 2395.246268] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2395.258300] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2395.365080] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2405.105926] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2410.979434] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2410.983339] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2410.988279] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2411.007095] Lustre: Skipped 2 previous similar messages [ 2411.046886] Lustre: Skipped 1 previous similar message [ 2414.266928] Lustre: server umount lustre-OST0001 complete [ 2416.099592] LustreError: 82616:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2416.126734] LustreError: 82616:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 2419.170796] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2419.178192] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2419.199214] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2424.686657] Lustre: server umount lustre-OST0000 complete [ 2430.441781] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2434.445357] Lustre: server umount lustre-MDT0000 complete [ 2436.597805] LustreError: 80792:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2436.622489] LustreError: 80792:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 2437.851358] LustreError: 83857:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789450341 with bad export cookie 13423448056144913255 [ 2437.855624] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2437.857988] LustreError: 83857:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 2438.223152] Lustre: server umount lustre-MDT0001 complete [ 2447.108852] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 01:32:29 (1789450349) [ 2448.971707] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2451.000818] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 01:32:32 (1789450352) [ 2452.558714] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2454.556731] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 01:32:36 (1789450356) [ 2456.227722] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2457.951390] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 01:32:39 (1789450359) [ 2459.701985] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2461.770318] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 01:32:43 (1789450363) [ 2463.632940] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2469.913347] loop: module loaded [ 2469.945508] loop0: detected capacity change from 0 to 4294967296 [ 2470.027420] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2471.913313] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2475.958485] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2480.719173] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2509.963183] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2510.017852] LustreError: 87983:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2510.027598] LustreError: 87983:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2510.032716] LustreError: 87983:0:(tgt_mount.c:2581:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2510.038742] LustreError: 87983:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2531.083781] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [ 2542.787550] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 2583.106933] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2583.349441] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2583.373980] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2583.434804] Lustre: lustre-MDT0000: new disk, initializing [ 2583.497833] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2583.502018] Lustre: Skipped 1 previous similar message [ 2583.509739] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2588.500488] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 2600.643950] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2600.817501] Lustre: 90852:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2601.180921] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2606.142391] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 2616.741871] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2623.439811] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2631.691286] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2631.975122] Lustre: 92820:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2638.455385] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2638.607648] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2639.362203] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 2650.083640] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2655.672614] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2655.928345] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2661.274535] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2661.502501] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2664.427538] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2664.428858] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2664.437156] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2664.437165] Lustre: Skipped 5 previous similar messages [ 2664.442324] LustreError: Skipped 1 previous similar message [ 2664.477314] Lustre: Skipped 4 previous similar messages [ 2669.315295] Lustre: server umount lustre-OST0000 complete [ 2679.299807] Lustre: server umount lustre-MDT0000 complete [ 2679.780755] LustreError: 90864:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2679.813121] LustreError: 90864:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 2683.135581] LustreError: 90845:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789450587 with bad export cookie 13423448056144913871 [ 2683.139325] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2683.149622] LustreError: 90845:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2683.499656] Lustre: server umount lustre-MDT0001 complete [ 2693.566556] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 01:36:35 (1789450595) [ 2700.584425] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2705.951952] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 2715.599444] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2715.996733] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2716.003832] Lustre: Skipped 3 previous similar messages [ 2720.705366] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 2728.554818] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2735.138693] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2742.069840] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2748.167798] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 2755.998206] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2760.569306] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2760.916557] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2765.495897] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2765.795770] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2773.200638] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2773.377491] Lustre: lustre-OST0001: new disk, initializing [ 2773.381866] Lustre: Skipped 2 previous similar messages [ 2773.386325] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2773.390441] Lustre: Skipped 3 previous similar messages [ 2773.394510] Lustre: 98607:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2779.730439] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2779.736608] Lustre: Skipped 2 previous similar messages [ 2779.738764] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2779.816405] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2780.053426] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 2789.773345] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2794.888793] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2795.072531] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2799.877817] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2800.166728] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2819.914894] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2846.183410] Lustre: Failing over lustre-MDT0000 [ 2846.574769] Lustre: server umount lustre-MDT0000 complete [ 2846.697125] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2846.699991] LustreError: 97534:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2846.713128] Lustre: Skipped 4 previous similar messages [ 2846.722073] LustreError: 97534:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 5 previous similar messages [ 2867.168180] Lustre: 66644:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789450755/real 1789450755] req@00000000dca8b2df x1876374142070656/t0(0) o400->MGC192.168.201.124@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1789450771 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2867.215269] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2870.283317] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2870.286141] LDISKFS-fs (dm-0): recovery complete [ 2870.340363] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2877.888345] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2880.269174] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2880.278759] Lustre: 95228:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2882.485468] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 2883.047841] Lustre: 95700:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2883.057323] Lustre: 95700:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 1 previous similar message [ 2883.078678] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2883.183540] LustreError: 101454:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2883.512179] LustreError: 101454:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2883.854112] LustreError: 101454:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2883.869523] LustreError: 101454:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 1 previous similar message [ 2884.192230] LustreError: 101454:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2884.199552] LustreError: 101454:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 1 previous similar message [ 2884.209544] Lustre: 101454:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2884.221808] Lustre: 101454:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 7 previous similar messages [ 2884.869299] LustreError: 101454:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2884.880742] LustreError: 101454:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 2 previous similar messages [ 2885.208131] LustreError: 101454:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2885.220094] LustreError: 101454:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 2 previous similar messages [ 2886.536381] Lustre: 101454:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2886.555471] Lustre: 101454:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 6 previous similar messages [ 2886.889579] LustreError: 101454:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2886.899211] LustreError: 101454:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 5 previous similar messages [ 2887.225403] LustreError: 101454:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2887.235676] LustreError: 101454:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 5 previous similar messages [ 2890.725553] Lustre: 101454:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2890.745702] Lustre: 101454:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 12 previous similar messages [ 2891.067334] LustreError: 101454:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2891.096756] LustreError: 101454:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 12 previous similar messages [ 2891.424168] LustreError: 101454:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2891.436322] LustreError: 101454:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 12 previous similar messages [ 2898.792767] Lustre: 101454:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2898.808345] Lustre: 101454:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 24 previous similar messages [ 2899.137103] LustreError: 101454:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2899.157081] LustreError: 101454:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 24 previous similar messages [ 2899.480193] LustreError: 101454:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2899.490492] LustreError: 101454:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 24 previous similar messages [ 2915.100469] Lustre: 101454:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2915.127171] Lustre: 101454:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 50 previous similar messages [ 2915.145705] LustreError: 101454:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2915.161833] LustreError: 101454:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 49 previous similar messages [ 2915.482242] LustreError: 101454:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2915.492855] LustreError: 101454:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 49 previous similar messages [ 2940.040694] Lustre: 101454:0:(ldlm_lib.c:2428:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2940.050066] Lustre: 101454:0:(ldlm_lib.c:2438:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2940.064657] Lustre: 101454:0:(genops.c:1600:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client ec15aa11-2cb9-45a3-971b-6b4a5c552ee2@192.168.201.24@tcp [ 2940.082055] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2940.095309] Lustre: 101454:0:(ldlm_lib.c:1948:abort_req_replay_queue()) @@@ aborted: req@00000000a768abab x1876374125089536/t0(8589934774) o35->ec15aa11-2cb9-45a3-971b-6b4a5c552ee2@192.168.201.24@tcp:439/0 lens 392/0 e 0 to 0 dl 1789450854 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2940.132201] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2940.146097] Lustre: lustre-MDT0000: Denying connection for new client ec15aa11-2cb9-45a3-971b-6b4a5c552ee2 (at 192.168.201.24@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2940.174295] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2940.217798] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2940.224057] Lustre: Skipped 2 previous similar messages [ 2940.224487] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2940.299754] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 2940.311347] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:78 to 0x280000401:577) [ 2956.772712] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2956.781975] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2956.799488] Lustre: Skipped 1 previous similar message [ 2956.815361] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2956.820306] Lustre: Skipped 3 previous similar messages [ 2962.686509] Lustre: server umount lustre-OST0000 complete [ 2972.892372] Lustre: server umount lustre-OST0001 complete [ 2982.507432] Lustre: server umount lustre-MDT0000 complete [ 2985.445188] LustreError: 95700:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2985.471934] LustreError: 95700:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 45 previous similar messages [ 2987.295196] LustreError: 95212:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789450891 with bad export cookie 13423448056144965090 [ 2987.313819] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2987.325357] LustreError: 95212:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2987.993436] Lustre: server umount lustre-MDT0001 complete [ 2998.963446] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 01:41:40 (1789450900) [ 3010.133506] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3010.159650] Lustre: *** cfs_fail_loc=197, val=0*** [ 3025.953405] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3031.073985] LustreError: 103434:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.124@tcp: failed processing log, type 4: rc = -110 [ 3036.334242] Lustre: *** cfs_fail_loc=197, val=0*** [ 3036.338162] Lustre: Skipped 65 previous similar messages [ 3056.864320] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3056.870891] Lustre: Skipped 3 previous similar messages [ 3063.825884] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 3072.956978] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3075.429568] Lustre: server umount lustre-OST0000 complete [ 3085.653562] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 01:43:07 (1789450987) [ 3108.817869] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3114.776419] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 3125.318078] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3129.800855] LustreError: 105085:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3129.807696] LustreError: 105085:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3129.818795] LustreError: 105085:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 20, retries 0, failed: rc = -5 [ 3130.158164] Lustre: server umount lustre-MDT0000 complete [ 3169.633611] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [ 3183.267935] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 3219.786950] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3220.010619] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3220.032406] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3220.118178] Lustre: lustre-MDT0000: new disk, initializing [ 3220.222773] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3224.638809] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 3234.930171] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3235.023624] Lustre: 109261:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3235.299385] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3240.483289] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 3250.059688] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3255.602357] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3264.481676] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3265.022663] Lustre: 111228:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3272.672540] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 3273.502454] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3287.103246] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3293.240574] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3293.485368] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3298.338534] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3298.510950] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3304.419239] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3304.440503] LustreError: Skipped 3 previous similar messages [ 3304.444347] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3304.452961] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3304.452969] Lustre: Skipped 9 previous similar messages [ 3304.502783] Lustre: Skipped 6 previous similar messages [ 3306.467894] Lustre: server umount lustre-OST0000 complete [ 3324.385496] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3324.612883] Lustre: server umount lustre-MDT0000 complete [ 3324.898373] LustreError: 109269:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3324.923673] LustreError: 109269:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 5 previous similar messages [ 3328.417478] LustreError: 111239:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789451232 with bad export cookie 13423448056144975723 [ 3328.443942] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3328.446756] LustreError: 111239:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3328.786873] Lustre: server umount lustre-MDT0001 complete [ 3337.590806] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 01:47:19 (1789451239) [ 3353.186547] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 3356.373312] Key type lgssc unregistered [ 3356.677653] LNet: 113740:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3356.686289] LNetError: 113740:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3356.736800] LNet: Removed LNI 192.168.201.124@tcp [ 3357.768051] Key type .llcrypt unregistered [ 3357.776100] Key type ._llcrypt unregistered [ 3377.535674] Key type ._llcrypt registered [ 3377.537781] Key type .llcrypt registered [ 3377.648707] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [ 3397.105714] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 3398.306392] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3398.471108] alg: No test for adler32 (adler32-zlib) [ 3399.523926] Lustre: Lustre: Build Version: 2.17.58_40_gc6c2479 [ 3399.768254] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 3401.432219] Key type lgssc registered [ 3402.324774] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3442.396801] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3442.430205] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3443.750302] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3443.795491] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3443.938248] Lustre: lustre-MDT0000: new disk, initializing [ 3444.055892] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3444.071677] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3448.179459] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 3459.039356] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3459.302195] Lustre: lustre-OST0000: new disk, initializing [ 3459.308301] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3459.314281] Lustre: lustre-OST0000: Not available for connect from 0@lo (not set up) [ 3459.317809] Lustre: Skipped 1 previous similar message [ 3459.325273] Lustre: 117912:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3459.382320] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3464.727553] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3464.734402] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3464.771036] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3465.430737] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 3475.840711] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3476.026344] Lustre: lustre-OST0001: new disk, initializing [ 3476.034113] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3476.042392] Lustre: 118772:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3476.141039] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3481.968735] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 3485.887161] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3485.897248] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3485.941248] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3491.298532] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3491.331092] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3496.427414] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3496.434128] Lustre: Skipped 1 previous similar message [ 3497.085953] Lustre: server umount lustre-MDT0000 complete [ 3503.299213] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3507.680854] LustreError: 118771:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789451411 with bad export cookie 5219170487650580692 [ 3507.687952] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3507.695493] LustreError: 118771:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3507.836792] Lustre: server umount lustre-OST0000 complete [ 3512.344981] Lustre: server umount lustre-OST0001 complete [ 3522.372530] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 01:50:24 (1789451424) [ 3543.737959] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 01:50:45 (1789451445) [ 3556.680794] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 3586.387507] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 01:51:27 (1789451487) [ 3624.811386] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [ 3635.157271] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 3671.387787] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3671.607911] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3671.627943] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3671.672230] Lustre: lustre-MDT0000: new disk, initializing [ 3671.704875] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3671.711970] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3676.397842] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 3689.815818] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3690.033347] Lustre: 126524:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3690.135190] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3690.142406] Lustre: Skipped 1 previous similar message [ 3690.276859] Lustre: lustre-MDT0001: new disk, initializing [ 3690.462706] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3690.510261] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3690.538783] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3697.353907] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 3708.770843] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3713.295609] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3719.533974] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3719.830230] Lustre: lustre-OST0000: new disk, initializing [ 3719.836889] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3719.845489] Lustre: 128498:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3719.958234] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3725.906357] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 3726.413773] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3726.428215] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3726.574353] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3735.814858] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3740.864621] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3741.049885] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3745.818301] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3746.064904] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3752.422748] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3752.425175] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3752.437221] LustreError: Skipped 1 previous similar message [ 3752.441449] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3752.441456] Lustre: Skipped 1 previous similar message [ 3752.465340] Lustre: Skipped 2 previous similar messages [ 3754.234348] Lustre: server umount lustre-OST0000 complete [ 3762.145115] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3762.152901] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3762.178107] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3762.180954] Lustre: Skipped 1 previous similar message [ 3772.384138] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3772.530619] Lustre: server umount lustre-MDT0000 complete [ 3776.472526] LustreError: 126517:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789451680 with bad export cookie 5219170487650581917 [ 3776.476965] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3776.484129] LustreError: 126517:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3776.772983] Lustre: server umount lustre-MDT0001 complete [ 3788.975394] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 01:54:50 (1789451690) [ 3801.919557] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3802.747484] LustreError: 130910:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3802.778370] LustreError: 130910:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 3802.847715] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3808.243239] LustreError: 130911:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3810.163277] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 3812.459423] LustreError: 130910:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3817.955873] LustreError: 130911:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3819.114657] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3824.321129] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 3832.840736] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3838.599476] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3845.547313] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3845.999601] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3846.004391] Lustre: Skipped 1 previous similar message [ 3852.309239] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 3861.199763] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3869.827681] Lustre: *** cfs_fail_loc=159, val=0*** [ 3886.448839] Lustre: lustre-MDT0000: Client d8662e4c-7ed9-4347-94d7-0e0f75976420 (at 192.168.201.24@tcp) reconnecting [ 3887.960872] Lustre: *** cfs_fail_loc=159, val=0*** [ 3887.963289] Lustre: Skipped 6 previous similar messages [ 3904.371138] Lustre: lustre-MDT0000: Client d8662e4c-7ed9-4347-94d7-0e0f75976420 (at 192.168.201.24@tcp) reconnecting [ 3910.638563] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3910.645531] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3910.660328] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3910.664213] Lustre: Skipped 2 previous similar messages [ 3910.688645] Lustre: Skipped 3 previous similar messages [ 3916.136952] Lustre: server umount lustre-OST0000 complete [ 3920.866375] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3920.887349] Lustre: Skipped 1 previous similar message [ 3925.995845] LustreError: 131640:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3926.328897] Lustre: server umount lustre-MDT0000 complete [ 3931.085624] LustreError: 130890:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789451834 with bad export cookie 5219170487650582799 [ 3931.087644] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3931.093479] LustreError: 130890:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3931.511119] Lustre: server umount lustre-MDT0001 complete [ 3944.385080] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 3947.158964] Key type lgssc unregistered [ 3947.390480] LNet: 134687:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3947.399465] LNetError: 134687:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3948.457468] LNet: Removed LNI 192.168.201.124@tcp [ 3949.585592] Key type .llcrypt unregistered [ 3949.590952] Key type ._llcrypt unregistered [ 3979.329381] Key type ._llcrypt registered [ 3979.333538] Key type .llcrypt registered [ 3979.406993] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 3980.583956] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3980.689054] alg: No test for adler32 (adler32-zlib) [ 3982.017729] Lustre: Lustre: Build Version: 2.17.58_40_gc6c2479 [ 3982.251503] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 3983.936216] Key type lgssc registered [ 3985.320878] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3993.764657] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3993.804947] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3995.484314] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4000.166385] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 4005.859762] LustreError: 136099:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4005.887240] LustreError: 136099:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4010.228911] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4010.654426] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4015.325991] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 4026.108913] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4033.064749] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4044.347127] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4045.136151] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4055.134658] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 4065.564735] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4065.589687] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4076.538419] Lustre: *** cfs_fail_loc=159, val=0*** [ 4092.735666] Lustre: lustre-MDT0000: Client 3512b06b-08a2-495d-9fea-3c744db6e8b9 (at 192.168.201.24@tcp) reconnecting [ 4095.124899] Lustre: *** cfs_fail_loc=159, val=0*** [ 4110.666896] Lustre: lustre-MDT0001: Client 3512b06b-08a2-495d-9fea-3c744db6e8b9 (at 192.168.201.24@tcp) reconnecting [ 4112.420053] Lustre: *** cfs_fail_loc=159, val=0*** [ 4112.426625] Lustre: Skipped 3 previous similar messages [ 4128.095761] Lustre: lustre-MDT0001: Client 3512b06b-08a2-495d-9fea-3c744db6e8b9 (at 192.168.201.24@tcp) reconnecting [ 4132.127437] Lustre: 139135:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4134.930475] Lustre: *** cfs_fail_loc=159, val=0*** [ 4134.942469] Lustre: Skipped 4 previous similar messages [ 4150.663567] Lustre: lustre-MDT0000: Client b930d205-71a1-4a1c-8d82-087cefbb3467 (at 192.168.201.24@tcp) reconnecting [ 4152.510212] Lustre: *** cfs_fail_loc=159, val=0*** [ 4152.513040] Lustre: Skipped 14 previous similar messages [ 4168.080360] Lustre: lustre-MDT0000: Client b930d205-71a1-4a1c-8d82-087cefbb3467 (at 192.168.201.24@tcp) reconnecting [ 4170.378806] Lustre: 139441:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4178.402401] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4178.412649] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4178.433310] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4178.444360] Lustre: Skipped 1 previous similar message [ 4179.302405] Lustre: server umount lustre-OST0000 complete [ 4185.057834] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4185.066587] LustreError: Skipped 1 previous similar message [ 4185.074053] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4185.091468] Lustre: Skipped 1 previous similar message [ 4185.101203] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4189.160096] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4189.167069] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4189.195866] Lustre: Skipped 1 previous similar message [ 4190.648953] Lustre: server umount lustre-MDT0000 complete [ 4194.273185] LustreError: 136093:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4194.315455] LustreError: 136093:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4194.636565] LustreError: 139255:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789452098 with bad export cookie 11564867416507208250 [ 4194.638818] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4194.654015] LustreError: 139255:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4194.854541] Lustre: server umount lustre-MDT0001 complete [ 4208.285714] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 4211.425431] Key type lgssc unregistered [ 4211.824268] LNet: 140498:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4211.838286] LNetError: 140498:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4211.862163] LNet: Removed LNI 192.168.201.124@tcp [ 4212.870334] Key type .llcrypt unregistered [ 4212.876165] Key type ._llcrypt unregistered [ 4242.833763] Key type ._llcrypt registered [ 4242.835579] Key type .llcrypt registered [ 4242.963212] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 4243.875198] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4243.904849] alg: No test for adler32 (adler32-zlib) [ 4245.185955] Lustre: Lustre: Build Version: 2.17.58_40_gc6c2479 [ 4245.416069] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 4247.153511] Key type lgssc registered [ 4248.290785] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4255.887519] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4255.928338] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4257.300686] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4262.588653] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 4267.491420] LustreError: 141930:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4267.513953] LustreError: 141930:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4272.479199] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4272.611371] LustreError: 141929:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4272.898771] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4277.818021] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 4287.032404] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4292.601094] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4300.528215] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4300.989309] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4303.064279] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4307.492394] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 4316.212548] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4317.466936] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4323.488780] Lustre: 144515:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4327.503420] Lustre: 144564:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4329.420213] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4330.341583] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4335.343205] Lustre: 144721:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4336.155803] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4337.066535] Lustre: 144819:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4343.274301] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4343.275777] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4343.285137] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4343.296651] Lustre: Skipped 1 previous similar message [ 4347.776596] Lustre: server umount lustre-OST0000 complete [ 4353.506186] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4353.509850] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4353.527894] Lustre: Skipped 1 previous similar message [ 4353.534751] Lustre: Skipped 1 previous similar message [ 4358.628946] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4358.640531] Lustre: Skipped 2 previous similar messages [ 4358.874609] Lustre: server umount lustre-MDT0000 complete [ 4363.071752] LustreError: 141911:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789452266 with bad export cookie 11120653984100050399 [ 4363.087945] LustreError: 141911:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4363.089880] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4363.527537] Lustre: server umount lustre-MDT0001 complete [ 4376.337990] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 4379.091843] Key type lgssc unregistered [ 4379.393642] LNet: 145873:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4379.415201] LNetError: 145873:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4379.432670] LNet: Removed LNI 192.168.201.124@tcp [ 4380.316236] Key type .llcrypt unregistered [ 4380.320196] Key type ._llcrypt unregistered [ 4408.257209] Key type ._llcrypt registered [ 4408.258531] Key type .llcrypt registered [ 4408.470837] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 4409.908198] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4410.002697] alg: No test for adler32 (adler32-zlib) [ 4411.102619] Lustre: Lustre: Build Version: 2.17.58_40_gc6c2479 [ 4411.340845] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 4413.048202] Key type lgssc registered [ 4414.355633] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4422.523959] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4422.555709] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4424.052542] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4429.156541] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 4433.411828] LustreError: 147285:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4433.443420] LustreError: 147285:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4438.502224] LustreError: 147286:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4439.271575] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4439.790338] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4445.106779] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 4455.590984] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4463.648232] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4472.657904] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4473.316472] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4480.599349] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 4484.614863] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4488.921230] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4493.850833] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4500.866390] Lustre: *** cfs_fail_loc=159, val=0*** [ 4517.193119] Lustre: lustre-MDT0000: Client 3659143d-1319-4d6a-9718-c5bd144b0f0a (at 192.168.201.24@tcp) reconnecting [ 4534.753466] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4534.763419] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4534.777852] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4534.784863] Lustre: Skipped 1 previous similar message [ 4534.814696] Lustre: Skipped 1 previous similar message [ 4539.883946] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4539.898551] Lustre: Skipped 1 previous similar message [ 4540.363934] Lustre: server umount lustre-OST0000 complete [ 4547.045256] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4547.050040] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4547.089203] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4550.113828] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4550.116221] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4552.137471] Lustre: server umount lustre-MDT0000 complete [ 4555.237157] LustreError: 148017:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4555.269546] LustreError: 148017:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4556.722442] LustreError: 147268:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789452460 with bad export cookie 2857933939926616678 [ 4556.728574] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4556.735298] LustreError: 147268:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4557.231624] Lustre: server umount lustre-MDT0001 complete [ 4572.480275] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 4576.250272] Key type lgssc unregistered [ 4576.694533] LNet: 150973:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4576.709095] LNetError: 150973:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4576.734465] LNet: Removed LNI 192.168.201.124@tcp [ 4577.954279] Key type .llcrypt unregistered [ 4577.961900] Key type ._llcrypt unregistered [ 4609.645424] Key type ._llcrypt registered [ 4609.647631] Key type .llcrypt registered [ 4609.783651] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 4611.317979] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4611.352278] alg: No test for adler32 (adler32-zlib) [ 4612.493933] Lustre: Lustre: Build Version: 2.17.58_40_gc6c2479 [ 4612.799213] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 4614.561220] Key type lgssc registered [ 4615.791190] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4625.083112] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4625.130790] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4626.657479] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4631.825562] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 4636.152208] LustreError: 152386:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4636.172071] LustreError: 152386:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4641.216095] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4641.249588] LustreError: 152387:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4641.609489] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4646.320739] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 4655.484290] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4661.878188] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4670.673865] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4671.209059] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4672.259559] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4677.389694] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 4685.522079] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4686.584494] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4706.927352] Lustre: lustre-OST0000: evicting 27297445-c10c-4e19-a63e-76cc1c7bdce5 (at 192.168.201.24@tcp) by administrative request [ 4706.941877] Lustre: Skipped 1 previous similar message [ 4738.020921] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4738.027626] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4738.034622] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4738.056740] Lustre: Skipped 1 previous similar message [ 4743.140625] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4743.157852] Lustre: Skipped 2 previous similar messages [ 4748.257319] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4750.304160] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4750.720468] Lustre: server umount lustre-OST0000 complete [ 4758.498432] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4758.522687] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4758.532295] Lustre: Skipped 2 previous similar messages [ 4760.982431] Lustre: server umount lustre-MDT0000 complete [ 4763.621634] LustreError: 155308:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4763.641423] LustreError: 155308:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4765.293223] LustreError: 152368:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789452669 with bad export cookie 8209671564253413551 [ 4765.299421] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4765.300021] LustreError: 152368:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4765.711261] Lustre: server umount lustre-MDT0001 complete [ 4780.599709] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 4783.575691] Key type lgssc unregistered [ 4784.015544] LNet: 156460:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4784.045269] LNetError: 156460:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4785.133350] LNet: Removed LNI 192.168.201.124@tcp [ 4786.529467] Key type .llcrypt unregistered [ 4786.532145] Key type ._llcrypt unregistered [ 4877.175768] Key type ._llcrypt registered [ 4877.177833] Key type .llcrypt registered [ 4877.288642] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 4879.000213] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4879.117252] alg: No test for adler32 (adler32-zlib) [ 4880.391299] Lustre: Lustre: Build Version: 2.17.58_40_gc6c2479 [ 4880.682830] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 4882.352772] Key type lgssc registered [ 4883.796365] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4892.473521] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4892.496483] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4894.175712] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4899.617670] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 4904.427398] LustreError: 158949:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4904.452319] LustreError: 158949:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4909.087465] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4909.500672] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4914.957635] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 4923.780127] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4931.055291] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4939.597510] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4940.229431] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4948.607081] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 4955.661875] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4957.594162] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4964.877796] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4980.199571] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4980.234913] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4984.294388] Lustre: server umount lustre-OST0000 complete [ 4990.445928] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4990.472031] Lustre: Skipped 1 previous similar message [ 4990.485287] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4990.502964] Lustre: Skipped 2 previous similar messages [ 4995.553324] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4995.573173] Lustre: Skipped 1 previous similar message [ 5000.675244] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5002.210320] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5002.637349] Lustre: server umount lustre-MDT0000 complete [ 5005.794990] LustreError: 158944:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5005.819614] LustreError: 158944:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 5006.755534] LustreError: 158929:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789452910 with bad export cookie 12896453005131211561 [ 5006.768412] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5007.150724] Lustre: server umount lustre-MDT0001 complete [ 5021.801440] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 5025.873913] Key type lgssc unregistered [ 5026.321267] LNet: 162515:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5026.331735] LNetError: 162515:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5027.367242] LNet: Removed LNI 192.168.201.124@tcp [ 5029.136145] Key type .llcrypt unregistered [ 5029.138762] Key type ._llcrypt unregistered [ 5056.022516] Key type ._llcrypt registered [ 5056.024703] Key type .llcrypt registered [ 5056.111190] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 5057.438204] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5057.565220] alg: No test for adler32 (adler32-zlib) [ 5058.701342] Lustre: Lustre: Build Version: 2.17.58_40_gc6c2479 [ 5058.932505] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 5060.608239] Key type lgssc registered [ 5061.628945] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5077.575623] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 02:16:19 (1789452979) [ 5098.813406] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [ 5112.298628] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 5154.172302] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5154.203975] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5155.528117] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5155.570840] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5155.690794] Lustre: lustre-MDT0000: new disk, initializing [ 5155.776974] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5155.791285] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5160.993878] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 5173.869377] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5174.051588] Lustre: 166782:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5174.111256] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5174.115076] Lustre: Skipped 1 previous similar message [ 5174.190786] Lustre: lustre-MDT0001: new disk, initializing [ 5174.306523] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5174.345414] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5174.361819] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5180.564191] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 5191.948073] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5199.294983] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5206.505551] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5206.819559] Lustre: lustre-OST0000: new disk, initializing [ 5206.823840] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5206.829185] Lustre: 168750:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5206.911394] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5213.805684] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 5215.338899] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5215.356474] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5215.488559] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5225.946953] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5241.314969] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5241.317683] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5241.333660] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5242.943670] Lustre: server umount lustre-OST0000 complete [ 5251.045664] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5251.053380] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5251.066538] Lustre: Skipped 1 previous similar message [ 5251.071351] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5251.077421] Lustre: Skipped 1 previous similar message [ 5253.319198] Lustre: server umount lustre-MDT0000 complete [ 5257.714792] LustreError: 167578:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789453161 with bad export cookie 7272968570830495195 [ 5257.715804] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5257.723572] LustreError: 167578:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5258.220919] Lustre: server umount lustre-MDT0001 complete [ 5272.051977] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 5275.415123] Key type lgssc unregistered [ 5275.774741] LNet: 170658:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5275.781356] LNetError: 170658:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5275.810308] LNet: Removed LNI 192.168.201.124@tcp [ 5277.082208] Key type .llcrypt unregistered [ 5277.084799] Key type ._llcrypt unregistered [ 5306.763345] Key type ._llcrypt registered [ 5306.766138] Key type .llcrypt registered [ 5306.904494] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 5308.656283] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5308.674405] alg: No test for adler32 (adler32-zlib) [ 5309.739407] Lustre: Lustre: Build Version: 2.17.58_40_gc6c2479 [ 5310.152394] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 5311.912552] Key type lgssc registered [ 5313.214173] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5322.490591] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5322.520834] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5324.181220] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5328.487491] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 5333.476568] LustreError: 172075:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5333.496904] LustreError: 172075:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 5337.969751] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5338.508244] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5343.556794] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 5351.988636] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5358.026720] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5366.577572] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5367.065380] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5369.096091] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5375.010069] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 5385.768523] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5394.988280] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5395.019165] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5395.045640] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5395.052325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.076564] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5396.681646] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5396.700958] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5396.722387] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5396.736433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.762675] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5398.165990] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5398.191287] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5398.200720] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5398.217674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.255486] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5399.401273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.400789] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5400.409054] Lustre: Skipped 1 previous similar message [ 5400.419936] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5400.432142] Lustre: Skipped 1 previous similar message [ 5400.437575] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5400.442091] Lustre: Skipped 1 previous similar message [ 5400.446464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.455464] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5400.462683] Lustre: Skipped 1 previous similar message [ 5402.221920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.387818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.253068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.988446] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5404.993833] Lustre: Skipped 3 previous similar messages [ 5405.004634] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5405.011978] Lustre: Skipped 3 previous similar messages [ 5405.014282] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5405.017039] Lustre: Skipped 3 previous similar messages [ 5405.019596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.026167] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5405.031425] Lustre: Skipped 3 previous similar messages [ 5406.004784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.293063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.431354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.388316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.487181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.450705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.424449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.292952] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5413.301817] Lustre: Skipped 7 previous similar messages [ 5413.329196] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5413.343331] Lustre: Skipped 7 previous similar messages [ 5413.349674] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5413.354344] Lustre: Skipped 7 previous similar messages [ 5413.357870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.365052] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5413.372334] Lustre: Skipped 7 previous similar messages [ 5414.301544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.151857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.119207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.102557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.204541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.964601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.917514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.739052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.744588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.600476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.487409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.207678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.173974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.018766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.076034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.940569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.924692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.843365] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5429.852182] Lustre: Skipped 17 previous similar messages [ 5429.864686] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5429.879322] Lustre: Skipped 17 previous similar messages [ 5429.885666] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5429.892518] Lustre: Skipped 17 previous similar messages [ 5429.900843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.918822] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5429.927569] Lustre: Skipped 17 previous similar messages [ 5430.861511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.646866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.715285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.670255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.721661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.748581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.714239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.742953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.679399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.612273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.718869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.996310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.042460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.145250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.321954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.485245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.495611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.018505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.334969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.659448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.598818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.581935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.664396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.607253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.123195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.258470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.078078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.375282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.566123] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5462.575106] Lustre: Skipped 28 previous similar messages [ 5462.605661] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5462.618606] Lustre: Skipped 28 previous similar messages [ 5462.633971] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5462.644877] Lustre: Skipped 28 previous similar messages [ 5462.651176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.730227] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5462.735962] Lustre: Skipped 28 previous similar messages [ 5464.067336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.393263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.618193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.701207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.816692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.157995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.215078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.328634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.450125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.269390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.107585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.005411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.955067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.852614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.892763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.784284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.682847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.628568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.533658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.341990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.378433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.445803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.477809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.454789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.519409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.421878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.308963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.302579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.318213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.143894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.059059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.024713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.933222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.971131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.857516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.863495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.055451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.991743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.157218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.524902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.744370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.797528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.230944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.263684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.855254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.930418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.880449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.051089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.367300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.894874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.093367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.173641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.293639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.256509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.168994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.238302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.221411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.119763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.100655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.194166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.284716] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5527.289802] Lustre: Skipped 60 previous similar messages [ 5527.322846] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5527.331728] Lustre: Skipped 60 previous similar messages [ 5527.334975] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5527.339522] Lustre: Skipped 60 previous similar messages [ 5527.345870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.356641] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5527.369300] Lustre: Skipped 60 previous similar messages [ 5528.408863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.418243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.606393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.014722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.365882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.593361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.567452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.490393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.521767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.744971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.928543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.983456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.093356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.119467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.090680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.231653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.125921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.047057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.146846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.263019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.498643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.818626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.035548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.525070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.921235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.304167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.589200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.977716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.078560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.141440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.933763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.004550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.988495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.343898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.317827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.291590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.128499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.235848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.998781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.295684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.318587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.037376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.864410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.887224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.074054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.024836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.396754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.894045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.404245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.000274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.360743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.548925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.068981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.164369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.256976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.114392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.120316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.235742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.371604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.376200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.498644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.431543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.547095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.455843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.713913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.732707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.009266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.223873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.752517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.062564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.363288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.476846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.527269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.674324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.994402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.150785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.235479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.153097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.170638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.253716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.411816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.338312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.400736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.461075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.475057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.707855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.659880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.719938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.748272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.757935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.727463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.003607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.177547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.041809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.000143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.101852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.157319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.385242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.526513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.484683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.487341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.440775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.622567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.573560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.735684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.853307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.959468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.410097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.644283] LustreError: 172095:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5649.665723] LustreError: 172095:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5649.668385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.626547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.063966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.526902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.056867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.211962] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5656.218919] Lustre: Skipped 113 previous similar messages [ 5656.246473] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5656.256189] Lustre: Skipped 113 previous similar messages [ 5656.260610] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5656.265766] Lustre: Skipped 113 previous similar messages [ 5656.270708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.285769] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5656.295488] Lustre: Skipped 113 previous similar messages [ 5657.417795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.844342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.998254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.912247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.050257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.018258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.174673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.133421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.293246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.729752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.820732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.724953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.835603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.811832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.103660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.164338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.339503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.553431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.621221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.845476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.216146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.084749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.248533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.692967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.162378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.065260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.296323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.644817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.621111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.124652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.603355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.921082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.953494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.196073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.526513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.359306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.484963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.412413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.150855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.678500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.917862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.186895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.174519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.059919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.889537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.920674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.033833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.175143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.366738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.554280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.595732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.713380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.756565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.739578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.745559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.767307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.942982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.946275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.107783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.254171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.340192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.211749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.330401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.321251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.392804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.463682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.473816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.421537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.763862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.820272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.734289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.771290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.770779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.773753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.651306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.551686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.483409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.620409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.502205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.345364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.054512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.971963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.063370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.174823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.050515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.128766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.056036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.222668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.235265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.400052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.498714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.462461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.521879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.747272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.937355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.037562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.953674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.825750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.754299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.819797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.972323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.091774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.928752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.224195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.494626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.558246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.837928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.176174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.533503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.945562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.288545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.569290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.010668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.254825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.335629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.525886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.866442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.152250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.433837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.025558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.320036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.595312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.655507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.717458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.032451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.288889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.247929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.370902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.541616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.731028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.874061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.970141] LustreError: 172095:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5804.978296] LustreError: 172095:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5804.999321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.294084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.454670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.395255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.543741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.793147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.718947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.893880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.094959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.360959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.833701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.173967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.198741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.380184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.314642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.266981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.552247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.862419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.837202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.061249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.185698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.231416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.259632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.506716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.564260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.528207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.404745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.383945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.438506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.758790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.033603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.997878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.194883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.432086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.523354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.801243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.820163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.053661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.221891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.400741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.501911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.667993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.772426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.032499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.952752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.875677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.039983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.168941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.199357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.124100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.370455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.223285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.269360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.190956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.487383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.457614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.642648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.778901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.847795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.033506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.189150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.393964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.699668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.933812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.329289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.252515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.346621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.513125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.510176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.805996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.003850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.006098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.254313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.352272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.590925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.958074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.108073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.328631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.463346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.367050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.533505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.742696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.966475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.407317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.983317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.151392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.279593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.675742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.985762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.273890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.444970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.415851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.969106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.248432] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5912.255313] Lustre: Skipped 224 previous similar messages [ 5912.279931] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5912.288277] Lustre: Skipped 224 previous similar messages [ 5912.292952] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5912.299035] Lustre: Skipped 224 previous similar messages [ 5912.303776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.314238] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5912.316860] Lustre: Skipped 224 previous similar messages [ 5913.632939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.116714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.450218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.702624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.944890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.339601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.639640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.668787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.802171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.088247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.190493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.226105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.336332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.431076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.280882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.518952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.836958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.785357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.846440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.940088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.988385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.155302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.344214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.378761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.506494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.666669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.784435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.038535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.228178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.344827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.592363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.589711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.468690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.332137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.367722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.515657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.571120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.543953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.609498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.547597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.864367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.098196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.286931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.428089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.563467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.563693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.753899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.864181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.947066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.120933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.026364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.225803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.385271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.308588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.430604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.447740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.713971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.014835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.912940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.750633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.867969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.377639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.591688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.750371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.054489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.450123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.599908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.643686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.836814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.043403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.153947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.104820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.093487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.117590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.201790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.049226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.224259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.295412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.067461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.967319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.918526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.013979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.014883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.205553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.211792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.271949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.588769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.235372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.574351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.871192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.005724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.103792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.146067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.193242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.390432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.816201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.254913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.717683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.722817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.308236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.683711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.911339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.231680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.207606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.990142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.325947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.177799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.373911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.398691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.848633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.158611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.570256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.284240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.457283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.118779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.426733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.729719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.655792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.163290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.629485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.858833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.139556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.505435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.748465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.997683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.388987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.806675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.354832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.553838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.676992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.762576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.058522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.268037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.310698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.181847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.202178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.436762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.373470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.592369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.606916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.809585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.877988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.858244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.954440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.234282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.562314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.886151] LustreError: 172095:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6087.898801] LustreError: 172095:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6087.919943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.099264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.085979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.057914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.889757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.021267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.263239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.415784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.356447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.440924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.659357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.861840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.189953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.380778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.398351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.318905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.474258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.508456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.716484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.919854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.867453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.259915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.372754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.627548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.922521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.385167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.705399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.934023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.965928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.131411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.395403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.532418] LustreError: 172095:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6123.551434] LustreError: 172095:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6123.584193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.502267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.598618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.137889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.596603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.629666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.843734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.268124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.412676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.523891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.661063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.777291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.916697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.165153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.157904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.218525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.412830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.409860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.565739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.570388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.808248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.892627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.885329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.033878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.181182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.220720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.375365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.250996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.481536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.891167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.224954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.715643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.853979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.236971] LustreError: 172095:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6162.253811] LustreError: 172095:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6162.258508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.510303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.981364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.269732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.502475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.562564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.509614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.661947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.773754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.786100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.083268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.127311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.302365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.421790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.727592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.096299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.399943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.600851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.783033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.906305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.159769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.631446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.992363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.541368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.973973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.090575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.206455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.362648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.376832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.567569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.834607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.141534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.328386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.589684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.823676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.058828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.438570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.851897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.271887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.800877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.098272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.139840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.330766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.542271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.508652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.803587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.977791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.836246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.912320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.506696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.821512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.999537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.609756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.113778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.506584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.656109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.892926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.098402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.277296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.437085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.778640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.831455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.010324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.134936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.361746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.695846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.772629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.656627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.575609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.425775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.668265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.708384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.764519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.802681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.302532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.453029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.494241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.493171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.774637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.771721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.917041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.947351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.984556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.024372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.159575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.280382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.427059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.749964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.014528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.180313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.588095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.999459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.223569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.512598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.812500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.911414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.028681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.111911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.978115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.165740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.341334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.639954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.929471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.993137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.125264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.225253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.427725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.510043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.644365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.783820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.888455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.954165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.961537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.827888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.936948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.725621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.804918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.820380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.757076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.832790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.929063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.047034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.114840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.116987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.370944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.514839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.598778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.804169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.892357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.935416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.079255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.318572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.571882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.604478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6319.664438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.505935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.362691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.080857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.070599] LustreError: 172095:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6323.085628] LustreError: 172095:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6323.090923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.175837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.191253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.059444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.978532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.969252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.822628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.075951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6331.426405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.336356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.465736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.437276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.523457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.426848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.407306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6338.442782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.591398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.525220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.569780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6342.492833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6343.574612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.483779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.707681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.637257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.990546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.071516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.045046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.998032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6351.958050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.889167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.922424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.051758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.890884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.812331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.786293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.654614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.652211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.778648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6361.745191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6362.880548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.732909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.886250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.792761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.153897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.331164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.247322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.371208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.534561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.638591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6373.551527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.527270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.476907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6377.004650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.346365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.735687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.128907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6382.445515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.638502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6385.088495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6386.075655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.290648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6388.328387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6389.576987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.042367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6392.206217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.329168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6394.320736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6395.796049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.137732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.334728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6399.525614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.549201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6401.655852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.659900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6403.569465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6404.563734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.574972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6406.569358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6407.638164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6408.730647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6409.752413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.841608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.723657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6412.733339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.729612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6414.654250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.582599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.428764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.453439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6418.565610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.405216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.327427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6421.201491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6422.250974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6423.256267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.346360] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6424.350223] Lustre: Skipped 443 previous similar messages [ 6424.365958] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6424.376400] Lustre: Skipped 443 previous similar messages [ 6424.386445] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6424.389359] Lustre: Skipped 443 previous similar messages [ 6424.395745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.405822] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6424.411881] Lustre: Skipped 443 previous similar messages [ 6425.404494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6426.394212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6427.396479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6428.345743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6429.417836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.396819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.321961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6432.375952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6433.441723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6434.270713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.368434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6436.259058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6437.160962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.147273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.932848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6439.921821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6440.901275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.840687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.903277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6443.991890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6444.974631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.775775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6446.644865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6447.450502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6448.304975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6449.213883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6450.233105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6450.937979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.931133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6452.955542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6454.033158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.051365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6456.339959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6457.380602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6458.420620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6459.453937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6460.378247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6461.447829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6462.395898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6463.310259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6464.130801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6465.179198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6465.967793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6466.746641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6467.608814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6468.749833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6469.586721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6470.616212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6471.524805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6472.453434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6473.273938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6474.318341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6475.114878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6476.177432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6477.113443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6478.216889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6479.112796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6480.522901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6481.694111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6482.832603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6483.921954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6484.855684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6485.879159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6486.738894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6487.794501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6488.805121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6489.671257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6490.612196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6491.555318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6492.538335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6493.669165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6494.665285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6495.645040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6496.623444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6497.583738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6498.629888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6499.643112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6500.512560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6501.640148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6502.591906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6503.599372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6504.598892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6505.721571] LustreError: 172095:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6505.737907] LustreError: 172095:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6505.761455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6506.908816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6507.850931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6509.056331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6510.028428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6511.003334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6512.169263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6513.267077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6514.345317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6515.321193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6983.334090] LustreError: 173899:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6983.360035] LustreError: 172095:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7662.049205] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7662.060657] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7662.073665] Lustre: Skipped 92 previous similar messages [ 7662.086744] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7664.103448] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7664.106757] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7664.111535] Lustre: Skipped 1 previous similar message [ 7666.209500] Lustre: server umount lustre-OST0000 complete [ 7673.316836] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7673.337828] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7677.600089] Lustre: server umount lustre-MDT0000 complete [ 7679.472973] LustreError: 172803:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7679.499834] LustreError: 172803:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 7681.864263] LustreError: 172054:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789455585 with bad export cookie 13691380820513233739 [ 7681.865614] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7681.874967] LustreError: 172054:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7682.340791] Lustre: server umount lustre-MDT0001 complete [ 7695.729554] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 7699.186144] Key type lgssc unregistered [ 7699.507820] LNet: 227363:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7699.528476] LNetError: 227363:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7700.591122] LNet: Removed LNI 192.168.201.124@tcp [ 7701.282244] Key type .llcrypt unregistered [ 7701.284459] Key type ._llcrypt unregistered [ 7731.479748] Key type ._llcrypt registered [ 7731.483735] Key type .llcrypt registered [ 7731.644863] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 7733.055632] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7733.202634] alg: No test for adler32 (adler32-zlib) [ 7734.457277] Lustre: Lustre: Build Version: 2.17.58_40_gc6c2479 [ 7734.864122] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 7736.704288] Key type lgssc registered [ 7738.527248] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7748.345779] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7748.363675] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7749.820033] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7755.301675] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 7759.348356] LustreError: 228785:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7759.380225] LustreError: 228785:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 7764.466060] LustreError: 228786:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7767.791560] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7768.303371] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7773.672827] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 7783.095990] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7790.350571] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7799.332619] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7799.647675] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7807.778528] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 7815.800156] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7820.166912] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7825.355216] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7827.438663] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7827.447428] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7827.453923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7827.468632] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7827.473540] LustreError: 228805:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7829.474076] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7829.477600] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7829.488479] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7830.504628] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7830.508623] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7831.852699] Lustre: lustre-OST0000: Not available for connect from 192.168.201.24@tcp (stopping) [ 7831.861713] Lustre: Skipped 1 previous similar message [ 7835.626492] LustreError: 230623:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7835.803241] Lustre: server umount lustre-OST0000 complete [ 7847.488795] LustreError: 228805:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7872.991519] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7873.587847] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7880.885771] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 7885.865804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7885.890233] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7889.912257] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7891.066992] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7891.091406] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 7899.861892] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 7 sec [ 7906.043575] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffffa0dc12bbd800.ost_server_uuid 50 [ 7907.939875] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffffa0dc12bbd800.ost_server_uuid in FULL state after 0 sec [ 7913.441304] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7913.447820] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7913.475338] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7915.446652] Lustre: server umount lustre-OST0000 complete [ 7922.149988] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7922.156443] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7922.179380] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7922.201162] Lustre: Skipped 1 previous similar message [ 7925.863731] Lustre: server umount lustre-MDT0000 complete [ 7927.268194] LustreError: 228786:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7927.299713] LustreError: 228786:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 7929.924057] LustreError: 228768:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789455833 with bad export cookie 17906755417224169208 [ 7929.925068] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7929.934596] LustreError: 228768:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7939.705351] Lustre: server umount lustre-MDT0001 complete [ 7953.052674] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 7956.339817] Key type lgssc unregistered [ 7956.750458] LNet: 234008:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7956.774177] LNetError: 234008:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7956.800183] LNet: Removed LNI 192.168.201.124@tcp [ 7957.964972] Key type .llcrypt unregistered [ 7957.967787] Key type ._llcrypt unregistered [ 7991.507768] Key type ._llcrypt registered [ 7991.510510] Key type .llcrypt registered [ 7991.611412] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 7992.696025] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7992.722067] alg: No test for adler32 (adler32-zlib) [ 7993.877057] Lustre: Lustre: Build Version: 2.17.58_40_gc6c2479 [ 7994.158465] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 7995.888214] Key type lgssc registered [ 7997.580455] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8008.450382] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8008.480739] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8010.063652] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8015.339408] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8019.431447] LustreError: 235736:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8019.448082] LustreError: 235736:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 8024.551873] LustreError: 235735:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8025.833135] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8026.352991] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8031.354221] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8034.895856] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 8043.671032] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8044.096214] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8053.514282] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8054.345990] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 8063.205944] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8063.394624] Lustre: lustre-OST0001: new disk, initializing [ 8063.399938] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8063.412455] Lustre: 238046:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8063.519799] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8071.190891] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8072.873033] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8072.885299] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8072.957984] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8083.621531] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8088.994251] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 8103.400508] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8103.407889] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8103.427575] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8103.908549] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8103.921613] Lustre: Skipped 1 previous similar message [ 8109.037739] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8109.049518] Lustre: Skipped 6 previous similar messages [ 8109.591106] Lustre: server umount lustre-MDT0000 complete [ 8113.058292] LustreError: 235715:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789456016 with bad export cookie 16187534479709171884 [ 8113.058764] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8113.071381] LustreError: 235715:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8113.400251] Lustre: server umount lustre-MDT0001 complete [ 8127.546927] Lustre: server umount lustre-OST0000 complete [ 8130.528519] Lustre: 234999:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789456018/real 1789456018] req@ffff8b7f291f6680 x1876380507794560/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789456034 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8130.554295] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8130.567609] Lustre: Skipped 1 previous similar message [ 8131.492784] Lustre: server umount lustre-OST0001 complete [ 8153.108436] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 8164.814539] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8165.298311] LustreError: 241147:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8165.397540] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8170.471115] LustreError: 241148:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8170.539649] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8175.588507] LustreError: 241147:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8179.139183] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8179.634370] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8183.432874] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8186.621092] Lustre: 242290:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8193.497478] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8200.135512] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8205.113711] LustreError: 242643:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8209.353745] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8209.670031] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8209.678017] Lustre: Skipped 1 previous similar message [ 8210.698360] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 8210.703510] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49760) [ 8216.343222] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8223.928359] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8228.141553] Lustre: 244161:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8241.122025] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8241.128744] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8241.147166] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8245.344940] Lustre: server umount mylustre-MDT0000 complete [ 8246.782269] LustreError: 244085:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8246.801096] LustreError: 244085:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 8250.076182] LustreError: 241892:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789456153 with bad export cookie 16187534479709174691 [ 8250.089981] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8250.096592] LustreError: 241892:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8250.632162] Lustre: server umount mylustre-MDT0001 complete [ 8264.315896] Lustre: server umount mylustre-OST0000 complete [ 8267.232209] Lustre: 234998:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789456155/real 1789456155] req@ffff8b7f28b7d880 x1876380507853440/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789456171 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8267.247202] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8267.253936] Lustre: Skipped 3 previous similar messages [ 8268.009365] Lustre: server umount mylustre-OST0001 complete [ 8289.851951] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 8299.158824] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8299.702466] LustreError: 246469:0:(ldlm_lib.c:1199:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8299.769573] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8304.401871] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8313.623857] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8319.535619] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8323.592534] Lustre: 247612:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8332.702484] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8342.540803] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8344.409491] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49792) [ 8352.833200] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8358.404791] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 8361.663084] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8370.356226] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8374.035784] Lustre: 249485:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8383.323898] Lustre: server umount tfs-MDT0000 complete [ 8383.978651] Lustre: tfs-MDT0000-lwp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8383.989622] LustreError: 248173:0:(ldlm_lib.c:1199:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8384.007884] Lustre: Skipped 3 previous similar messages [ 8384.036884] LustreError: 248173:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 10 previous similar messages [ 8387.144418] LustreError: 247213:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789456291 with bad export cookie 16187534479709176581 [ 8387.149732] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8387.166428] LustreError: 247213:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8387.514902] Lustre: server umount tfs-MDT0001 complete [ 8402.177794] Lustre: server umount tfs-OST0000 complete [ 8404.448526] Lustre: 234997:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789456292/real 1789456292] req@ffff8b7f1bce1880 x1876380507906176/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789456308 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8404.472100] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8406.092519] Lustre: server umount tfs-OST0001 complete [ 8434.328777] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 8447.141781] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8447.857493] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8447.868616] Lustre: Skipped 3 previous similar messages [ 8452.820599] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8462.775037] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8468.281509] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8471.574614] Lustre: 252878:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8478.617979] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8486.104177] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8492.444721] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49824) [ 8495.372420] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8500.746866] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 8504.269951] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8515.607030] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8521.483720] Lustre: 254748:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8528.151666] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 03:13:50 (1789456430) [ 8546.786519] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8546.807276] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8546.822702] Lustre: Skipped 2 previous similar messages [ 8546.836951] Lustre: Skipped 6 previous similar messages [ 8551.905278] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8551.921162] Lustre: Skipped 3 previous similar messages [ 8552.689206] Lustre: server umount lustre-MDT0000 complete [ 8556.890654] LustreError: 251713:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789456460 with bad export cookie 16187534479709178247 [ 8556.897979] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8556.907988] LustreError: 251713:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8557.025692] LustreError: 251733:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8557.044653] LustreError: 251733:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 8 previous similar messages [ 8561.067034] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8561.070182] Lustre: Skipped 2 previous similar messages [ 8571.360151] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8571.863916] Lustre: server umount lustre-MDT0001 complete [ 8582.389307] Lustre: server umount lustre-OST0000 complete [ 8592.546635] Lustre: server umount lustre-OST0001 complete [ 8600.751518] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [ 8614.075718] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 8658.448286] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8658.859858] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8658.931306] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8659.002357] Lustre: lustre-MDT0000: new disk, initializing [ 8659.113493] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 8659.127133] Lustre: Skipped 3 previous similar messages [ 8659.153421] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8664.056134] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8676.365964] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8676.588778] Lustre: 257938:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8676.768500] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8676.780292] Lustre: Skipped 1 previous similar message [ 8677.000757] Lustre: lustre-MDT0001: new disk, initializing [ 8677.228405] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8677.260568] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8682.881322] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8695.881345] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8696.291535] Lustre: lustre-OST0000: new disk, initializing [ 8696.297839] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8696.308691] Lustre: 259521:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8698.303295] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8698.313934] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8698.445467] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8705.595378] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8718.806967] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8718.971624] Lustre: lustre-OST0001: new disk, initializing [ 8718.976753] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8718.981956] Lustre: 260393:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8720.270189] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8720.281956] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8720.391556] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8727.335399] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8743.234104] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 03:17:24 (1789456644) [ 8761.953733] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 03:17:43 (1789456663) [ 8780.258132] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8780.264465] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8780.271343] Lustre: Skipped 3 previous similar messages [ 8780.281405] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8780.284290] Lustre: Skipped 5 previous similar messages [ 8784.229685] Lustre: server umount lustre-OST0000 complete [ 8789.985891] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8789.996853] LustreError: Skipped 1 previous similar message [ 8803.296134] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8803.482618] Lustre: server umount lustre-MDT0000 complete [ 8807.393821] LustreError: 257932:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789456711 with bad export cookie 16187534479709179801 [ 8807.405995] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8807.408065] LustreError: 257932:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 8812.513784] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8812.518176] Lustre: Skipped 13 previous similar messages [ 8813.994260] Lustre: server umount lustre-MDT0001 complete [ 8824.970945] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 8898.954597] Lustre: server umount lustre-OST0001 complete [ 8905.550382] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [ 8920.443857] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 8962.976846] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8963.293668] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8963.334604] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8963.456890] Lustre: lustre-MDT0000: new disk, initializing [ 8963.567734] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8963.573451] Lustre: Skipped 3 previous similar messages [ 8963.585805] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8968.654967] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8982.898668] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8983.141360] Lustre: 265246:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8983.275525] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8983.279341] Lustre: Skipped 1 previous similar message [ 8983.399503] Lustre: lustre-MDT0001: new disk, initializing [ 8983.653830] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8983.667957] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8989.695762] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 9003.048038] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9011.223321] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9019.937654] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9020.346313] Lustre: lustre-OST0000: new disk, initializing [ 9020.349647] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9020.356263] Lustre: 267217:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9022.189948] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9022.210850] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9022.437241] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9027.943198] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 9039.433693] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9048.036419] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9048.044181] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9048.052404] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9048.054988] Lustre: Skipped 6 previous similar messages [ 9048.078661] Lustre: Skipped 1 previous similar message [ 9053.316778] Lustre: server umount lustre-OST0000 complete [ 9063.540952] Lustre: server umount lustre-MDT0000 complete [ 9067.424838] LustreError: 265240:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789456971 with bad export cookie 16187534479709181271 [ 9067.430274] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9067.438578] LustreError: 265240:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9067.923693] Lustre: server umount lustre-MDT0001 complete [ 9080.182307] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 9082.708388] Key type lgssc unregistered [ 9083.039084] LNet: 268975:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9083.051383] LNetError: 268975:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9083.063755] LNet: Removed LNI 192.168.201.124@tcp [ 9083.974403] Key type .llcrypt unregistered [ 9083.978475] Key type ._llcrypt unregistered [ 9112.087243] Key type ._llcrypt registered [ 9112.088807] Key type .llcrypt registered [ 9112.180173] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 9112.897945] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9112.997769] alg: No test for adler32 (adler32-zlib) [ 9114.038547] Lustre: Lustre: Build Version: 2.17.58_40_gc6c2479 [ 9114.244439] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 9115.937627] Key type lgssc registered [ 9117.372781] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9126.775348] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9126.830575] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9128.385126] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9134.592888] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 9138.666555] LustreError: 270434:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9138.701647] LustreError: 270434:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 9139.689148] LustreError: 270435:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9144.802230] LustreError: 270434:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9149.921956] LustreError: 270435:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9152.931176] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9154.068406] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9161.658187] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 9170.912961] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9179.482568] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9188.781049] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9189.492503] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9196.960981] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 9199.807548] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 9205.472344] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9206.516831] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 9209.313188] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9209.325948] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9209.359246] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9210.338388] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9210.345453] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9210.345912] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9210.372467] Lustre: Skipped 1 previous similar message [ 9214.633919] Lustre: server umount lustre-OST0000 complete [ 9220.580926] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9220.583265] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9220.607547] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9225.702152] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9225.711267] Lustre: Skipped 2 previous similar messages [ 9230.817525] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9230.825025] Lustre: Skipped 1 previous similar message [ 9232.352230] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9232.590508] Lustre: server umount lustre-MDT0000 complete [ 9235.942563] LustreError: 271878:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9235.961946] LustreError: 271878:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 9236.369598] LustreError: 270416:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789457140 with bad export cookie 13219095502593054264 [ 9236.378199] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9236.385427] LustreError: 270416:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9236.709731] Lustre: server umount lustre-MDT0001 complete [ 9247.798955] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 9250.462980] Key type lgssc unregistered [ 9250.708182] LNet: 274022:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9250.712927] LNetError: 274022:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9250.728298] LNet: Removed LNI 192.168.201.124@tcp [ 9251.848430] Key type .llcrypt unregistered [ 9251.850400] Key type ._llcrypt unregistered [ 9271.104983] Key type ._llcrypt registered [ 9271.106501] Key type .llcrypt registered [ 9271.299968] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 9272.565605] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9272.689127] alg: No test for adler32 (adler32-zlib) [ 9273.814735] Lustre: Lustre: Build Version: 2.17.58_40_gc6c2479 [ 9274.052589] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 9275.705975] Key type lgssc registered [ 9276.905437] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9286.100528] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9286.121312] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9301.728840] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 9306.848962] LustreError: 275213:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.124@tcp: failed processing log, type 4: rc = -110 [ 9333.536404] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9340.329844] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 9348.274363] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9350.460215] Lustre: server umount lustre-OST0000 complete [ 9365.519427] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 9368.137860] Key type lgssc unregistered [ 9368.446916] LNet: 276844:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9368.466936] LNetError: 276844:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9368.484218] LNet: Removed LNI 192.168.201.124@tcp [ 9369.260392] Key type .llcrypt unregistered [ 9369.263390] Key type ._llcrypt unregistered [ 9411.271982] Key type ._llcrypt registered [ 9411.273852] Key type .llcrypt registered [ 9411.363912] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 9412.643330] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9412.733252] alg: No test for adler32 (adler32-zlib) [ 9413.912952] Lustre: Lustre: Build Version: 2.17.58_40_gc6c2479 [ 9414.173765] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 9415.824311] Key type lgssc registered [ 9416.941238] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9430.609149] loop0: detected capacity change from 0 to 204800000 [ 9430.801850] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9430.814445] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9431.297704] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9432.543315] loop0: detected capacity change from 0 to 204800000 [ 9432.703655] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9432.721221] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9433.101744] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9434.323753] loop0: detected capacity change from 0 to 204800000 [ 9434.564962] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9434.605391] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9434.741175] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9435.891872] loop0: detected capacity change from 0 to 204800000 [ 9436.028774] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9436.039757] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9436.183991] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9437.358738] loop0: detected capacity change from 0 to 204800000 [ 9437.388958] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9441.616836] loop0: detected capacity change from 0 to 204800000 [ 9441.648863] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9445.678834] loop0: detected capacity change from 0 to 204800000 [ 9445.707087] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9449.926980] loop0: detected capacity change from 0 to 204800000 [ 9449.957026] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9454.449980] loop0: detected capacity change from 0 to 204800000 [ 9454.556928] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9454.608899] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9454.688345] Lustre: 280255:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9454.697907] Lustre: 280255:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 9454.710280] Lustre: 280255:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 9454.726310] Lustre: 280255:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9454.741352] Lustre: 280255:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9454.755293] Lustre: 280255:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9454.767853] Lustre: 280255:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 9454.789312] Lustre: 280255:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 9454.797337] Lustre: 280255:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 9454.806388] Lustre: 280255:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 9454.826979] Lustre: 280255:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 9454.839260] Lustre: 280255:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 9454.849493] Lustre: 280255:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 9454.863411] Lustre: 280255:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 9460.522994] Lustre: server umount MGS complete [ 9461.628373] loop0: detected capacity change from 0 to 204800000 [ 9461.720138] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9462.111680] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9462.143829] Lustre: lustre-MDT0000: Aborting client recovery [ 9462.889202] loop1: detected capacity change from 0 to 204800000 [ 9463.114254] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9463.189880] Lustre: 280688:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9463.202276] Lustre: 280688:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 9463.209481] Lustre: 280688:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9463.217881] Lustre: 280688:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9463.225921] Lustre: 280688:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9463.233300] Lustre: 280688:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 9463.239844] Lustre: 280688:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 9463.257994] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9463.389196] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9463.434236] Lustre: lustre-MDT0001: Aborting client recovery [ 9464.266580] loop2: detected capacity change from 0 to 204800000 [ 9464.464734] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9464.558766] Lustre: 280766:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9464.575721] Lustre: 280766:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9464.586578] Lustre: 280766:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9464.599030] Lustre: 280766:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9464.612489] Lustre: 280766:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9464.638728] Lustre: 280766:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9464.649739] Lustre: 280766:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9464.786133] Lustre: Found index 0 for lustre-OST0000, updating log [ 9464.848618] Lustre: 280784:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9464.976706] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9464.997252] Lustre: lustre-OST0000: Aborting recovery [ 9465.953677] loop3: detected capacity change from 0 to 204800000 [ 9466.136471] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9466.196644] Lustre: 280850:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9466.216679] Lustre: 280850:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9466.234879] Lustre: 280850:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9466.254671] Lustre: 280850:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9466.276426] Lustre: 280850:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9466.287499] Lustre: 280850:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9466.298406] Lustre: 280850:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9466.314445] Lustre: Found index 1 for lustre-OST0001, updating log [ 9466.337508] Lustre: 280856:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9466.419802] Lustre: lustre-OST0001: Aborting recovery [ 9471.498442] LustreError: 280631:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 9, retries 0, failed: rc = -108 [ 9472.495578] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9472.500063] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9472.530252] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 9472.587578] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 9473.511226] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 9473.512136] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 9475.177483] LustreError: 280598:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 3651us: rc = -2 [ 9475.694993] LustreError: 280598:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 521us: rc = -2 [ 9475.711450] LustreError: 280598:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 17 previous similar messages [ 9488.864901] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9488.866163] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9488.866603] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9488.891436] Lustre: Skipped 2 previous similar messages [ 9493.991429] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9494.002493] Lustre: Skipped 6 previous similar messages [ 9494.513896] Lustre: server umount lustre-MDT0000 complete [ 9495.481592] LustreError: 280580:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789457399 with bad export cookie 907960063645856126 [ 9495.488558] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9495.505870] LustreError: 280580:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9499.106381] LustreError: 280596:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9499.107352] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9499.128418] LustreError: 280596:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 9499.156397] Lustre: Skipped 1 previous similar message [ 9499.161455] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9499.169775] Lustre: Skipped 1 previous similar message [ 9500.129072] LustreError: 280596:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9500.144577] LustreError: 280596:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 9501.944339] Lustre: server umount lustre-MDT0001 complete [ 9509.091977] Lustre: server umount lustre-OST0000 complete [ 9516.584652] Lustre: server umount lustre-OST0001 complete [ 9525.294768] Lustre: DEBUG MARKER: == conf-sanity test 108c: mkfs.lustre --mmp option enables MMP feature ========================================================== 03:30:26 (1789457426) [ 9548.398933] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 9590.127054] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9590.462718] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9590.498340] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9590.581399] Lustre: lustre-MDT0000: new disk, initializing [ 9590.665635] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9590.671570] Lustre: Skipped 1 previous similar message [ 9590.693446] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9596.031867] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 9603.084456] Lustre: server umount lustre-MDT0000 complete [ 9624.597626] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 03:32:06 (1789457526) [ 9655.811783] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [ 9667.881832] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 9703.969779] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9704.198779] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9704.219171] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9704.224252] Lustre: Skipped 1 previous similar message [ 9704.305396] Lustre: lustre-MDT0000: new disk, initializing [ 9704.419726] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9704.444407] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9709.159974] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 9721.752151] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9721.914449] Lustre: 287970:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9721.993698] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9721.997041] Lustre: Skipped 1 previous similar message [ 9722.080648] Lustre: lustre-MDT0001: new disk, initializing [ 9722.199576] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9722.241329] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9722.255854] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9726.911599] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 9736.345230] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9741.877931] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9748.244743] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9748.614948] Lustre: lustre-OST0000: new disk, initializing [ 9748.621553] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9748.629171] Lustre: 289935:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9748.714687] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9754.900105] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 9755.269408] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9755.281902] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9755.386323] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9764.413526] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9769.422378] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9782.518675] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9788.736422] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9842.665984] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9842.684625] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9842.695504] Lustre: Skipped 1 previous similar message [ 9842.710931] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9842.737237] Lustre: Skipped 3 previous similar messages [ 9847.780169] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9847.790184] Lustre: Skipped 1 previous similar message [ 9848.745845] Lustre: server umount lustre-OST0000 complete [ 9852.902493] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9852.922911] Lustre: Skipped 2 previous similar messages [ 9858.017942] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9858.030647] Lustre: Skipped 3 previous similar messages [ 9867.232550] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9868.258267] LustreError: 287975:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9868.275092] LustreError: 287975:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 9869.538797] Lustre: server umount lustre-MDT0000 complete [ 9872.353514] LustreError: 287978:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9872.363845] LustreError: 287978:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 9873.187239] LustreError: 287959:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789457777 with bad export cookie 907960063645859108 [ 9873.195842] LustreError: 287959:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9873.196988] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9873.502504] Lustre: server umount lustre-MDT0001 complete [ 9883.470634] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9888.573379] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 9893.420133] Lustre: server umount MGS complete [ 9905.550210] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9906.274286] LustreError: 293854:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9906.315565] LustreError: 293867:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9906.410190] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9911.037592] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 9916.900249] LustreError: 293855:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9916.924968] LustreError: 293855:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 9920.966118] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9925.606546] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 9934.043178] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9941.365577] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9949.468916] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9958.184957] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 9967.868705] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9997.286504] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9997.306262] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9997.327257] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9997.336694] Lustre: Skipped 1 previous similar message [ 9997.352961] Lustre: Skipped 3 previous similar messages [10000.599187] Lustre: server umount lustre-OST0000 complete [10007.543567] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10007.563307] Lustre: Skipped 1 previous similar message [10019.296388] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10019.474583] Lustre: server umount lustre-MDT0000 complete [10022.881206] LustreError: 293855:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10022.898398] LustreError: 293855:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [10023.355891] LustreError: 293835:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789457927 with bad export cookie 907960063645860942 [10023.364459] LustreError: 293835:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10023.380522] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10023.775326] Lustre: server umount lustre-MDT0001 complete [10034.777107] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [10037.743404] Key type lgssc unregistered [10038.069376] LNet: 297804:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10038.076485] LNetError: 297804:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10038.100209] LNet: Removed LNI 192.168.201.124@tcp [10039.050761] Key type .llcrypt unregistered [10039.052456] Key type ._llcrypt unregistered [10078.680784] Key type ._llcrypt registered [10078.686183] Key type .llcrypt registered [10078.804373] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [10095.315686] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [10096.656873] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10096.756068] alg: No test for adler32 (adler32-zlib) [10097.992838] Lustre: Lustre: Build Version: 2.17.58_40_gc6c2479 [10098.302329] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [10100.160358] Key type lgssc registered [10102.016874] Lustre: Echo OBD driver; http://www.lustre.org/ [10145.783694] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10145.817978] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10147.011247] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10147.038306] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10147.113744] Lustre: lustre-MDT0000: new disk, initializing [10147.240265] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10147.277373] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10152.014427] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [10163.509514] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10163.630435] Lustre: 301761:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10163.700330] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10163.704604] Lustre: Skipped 1 previous similar message [10163.780460] Lustre: lustre-MDT0001: new disk, initializing [10163.852630] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10163.878982] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10163.899245] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10168.285789] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [10178.708516] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10186.219749] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10194.355716] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10194.630720] Lustre: lustre-OST0000: new disk, initializing [10194.634507] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10194.645073] Lustre: 303734:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10194.769170] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10200.924286] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [10203.755278] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10203.764336] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10203.872817] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10211.554091] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10219.015583] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10228.991544] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10235.394524] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10280.933069] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10280.933885] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10280.940492] LustreError: Skipped 1 previous similar message [10280.941208] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10280.965095] Lustre: Skipped 1 previous similar message [10282.332452] Lustre: server umount lustre-OST0000 complete [10291.170567] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10291.188917] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10291.203378] Lustre: Skipped 2 previous similar messages [10293.951354] Lustre: server umount lustre-MDT0000 complete [10297.939160] LustreError: 304936:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789458201 with bad export cookie 10028814455486273364 [10297.944038] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10297.950070] LustreError: 304936:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10298.323208] Lustre: server umount lustre-MDT0001 complete [10307.932956] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10313.214305] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [10318.595383] Lustre: server umount MGS complete [10331.178790] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10331.711161] LustreError: 307547:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10331.728508] LustreError: 307547:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [10331.739968] LustreError: 307560:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10331.802491] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10335.759454] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [10337.251208] LustreError: 307548:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10341.368212] LustreError: 307547:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10345.156434] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10345.562424] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10350.459574] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [10358.191361] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10364.161194] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10370.550384] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10370.901494] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10377.390166] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [10385.278210] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10418.145918] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10418.170636] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10418.177019] Lustre: Skipped 2 previous similar messages [10418.201036] Lustre: Skipped 1 previous similar message [10423.273328] LustreError: 309371:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10423.309134] LustreError: 309371:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [10423.515769] Lustre: server umount lustre-OST0000 complete [10432.500516] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10432.514422] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10432.542421] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10434.941171] Lustre: server umount lustre-MDT0000 complete [10438.633415] LustreError: 307548:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10438.651035] LustreError: 307548:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [10439.318099] LustreError: 307529:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789458343 with bad export cookie 10028814455486275184 [10439.331265] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10439.340149] LustreError: 307529:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10440.006897] Lustre: server umount lustre-MDT0001 complete [10456.354810] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [10460.673924] Key type lgssc unregistered [10461.032990] LNet: 311493:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10461.049970] LNetError: 311493:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10461.086111] LNet: Removed LNI 192.168.201.124@tcp [10462.914946] Key type .llcrypt unregistered [10462.918977] Key type ._llcrypt unregistered [10488.766962] Key type ._llcrypt registered [10488.769894] Key type .llcrypt registered [10488.966488] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [10504.461844] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [10505.760227] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10506.021791] alg: No test for adler32 (adler32-zlib) [10506.997619] Lustre: Lustre: Build Version: 2.17.58_40_gc6c2479 [10507.181867] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [10508.800563] Key type lgssc registered [10509.796821] Lustre: Echo OBD driver; http://www.lustre.org/ [10535.066338] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [10553.198492] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10553.225589] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10554.639398] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10554.709919] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10554.857723] Lustre: lustre-MDT0000: new disk, initializing [10554.984207] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10555.013675] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10559.971661] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [10571.125287] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10571.580901] Lustre: lustre-OST0000: new disk, initializing [10571.589447] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10571.599371] Lustre: Skipped 1 previous similar message [10571.615479] Lustre: 315543:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10571.744624] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10578.549078] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [10578.553567] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [10578.604647] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [10580.304921] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [11490.791113] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11490.816459] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11493.588870] Lustre: server umount lustre-MDT0000 complete [11498.026633] LustreError: 314730:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789459401 with bad export cookie 8477638994755640874 [11498.032203] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11498.047533] LustreError: 314730:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11498.341590] Lustre: server umount lustre-OST0000 complete [11531.468092] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [11548.221909] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [11591.038962] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11591.347854] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11591.386606] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11591.488224] Lustre: lustre-MDT0000: new disk, initializing [11591.567908] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11591.586651] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11597.114735] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [11610.196286] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11610.350449] Lustre: 319924:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11610.421821] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11610.426524] Lustre: Skipped 1 previous similar message [11610.491898] Lustre: lustre-MDT0001: new disk, initializing [11610.546782] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11610.562487] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11610.566715] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11616.695354] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [11627.387493] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11634.561370] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11642.025633] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11642.228589] Lustre: lustre-OST0000: new disk, initializing [11642.231285] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11642.235539] Lustre: 321894:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11642.305425] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11649.540945] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [11649.629927] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11649.643774] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11649.883798] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11659.942267] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11665.859589] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11666.176345] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11671.500202] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11671.803515] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11675.618380] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11675.633873] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11675.638409] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11675.657859] Lustre: Skipped 1 previous similar message [11675.697307] Lustre: Skipped 1 previous similar message [11680.322029] Lustre: server umount lustre-OST0000 complete [11685.858652] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11685.865380] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11685.870574] Lustre: Skipped 1 previous similar message [11685.896177] Lustre: Skipped 1 previous similar message [11690.980112] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11690.987897] Lustre: Skipped 1 previous similar message [11696.099286] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11696.121180] Lustre: Skipped 1 previous similar message [11700.192327] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11700.502154] Lustre: server umount lustre-MDT0000 complete [11704.188798] LustreError: 323434:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789459608 with bad export cookie 8477638994758581700 [11704.200704] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11704.695736] Lustre: server umount lustre-MDT0001 complete [11709.519597] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [11711.633402] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 04:06:53 (1789459613) [11719.161167] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11719.642939] LustreError: 324210:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11719.658729] LustreError: 324210:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [11719.736569] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11724.779725] LustreError: 324211:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11724.984261] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [11729.899181] LustreError: 324210:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11734.478820] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11734.890600] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11739.748960] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [11747.151951] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11752.957909] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11759.368103] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11759.598864] LustreError: 326029:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11759.760791] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11766.046391] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [11774.652287] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11782.551771] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11782.788733] Lustre: lustre-OST0001: new disk, initializing [11782.792488] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [11782.796977] Lustre: 327047:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11782.900357] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11789.627775] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [11790.978772] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [11790.995744] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [11791.056064] LustreError: 327469:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11792.099334] LustreError: 327476:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11792.118944] LustreError: 327476:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [11793.131491] LustreError: 327469:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11795.238139] LustreError: 327476:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11795.254469] LustreError: 327476:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 4 previous similar messages [11799.330794] LustreError: 327469:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11799.350946] LustreError: 327469:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 6 previous similar messages [11800.995277] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11801.284328] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11807.588556] LustreError: 327469:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11807.611015] LustreError: 327469:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [11807.685554] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffffa0dc02ce9800.ost_server_uuid 50 [11809.630519] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffffa0dc02ce9800.ost_server_uuid in FULL state after 0 sec [11816.433365] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffffa0dc02ce9800.ost_server_uuid 50 [11818.240559] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffffa0dc02ce9800.ost_server_uuid in IDLE state after 0 sec [11820.095737] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [11835.363938] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11835.367438] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11835.372309] LustreError: Skipped 2 previous similar messages [11835.373298] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11835.402684] Lustre: Skipped 1 previous similar message [11840.285137] Lustre: server umount lustre-OST0001 complete [11841.826716] LustreError: 326330:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.201.24@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11852.273311] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11852.297803] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11852.326933] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11852.336701] Lustre: Skipped 5 previous similar messages [11852.781769] LustreError: 326330:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11852.822746] LustreError: 326330:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 5 previous similar messages [11854.284948] Lustre: server umount lustre-OST0000 complete [11863.014534] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11863.015931] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11863.038615] Lustre: Skipped 1 previous similar message [11863.042651] LustreError: Skipped 1 previous similar message [11865.952954] Lustre: server umount lustre-MDT0000 complete [11869.995429] LustreError: 324192:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789459773 with bad export cookie 8477638994758582582 [11870.004449] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11870.014904] LustreError: 324192:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [11870.503180] Lustre: server umount lustre-MDT0001 complete [11885.318785] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [11888.489896] Key type lgssc unregistered [11888.931913] LNet: 329736:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11888.938284] LNetError: 329736:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11888.955090] LNet: Removed LNI 192.168.201.124@tcp [11889.988606] Key type .llcrypt unregistered [11889.992649] Key type ._llcrypt unregistered [11919.977157] Key type ._llcrypt registered [11919.980871] Key type .llcrypt registered [11920.110712] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [11921.447101] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11921.557624] alg: No test for adler32 (adler32-zlib) [11922.815613] Lustre: Lustre: Build Version: 2.17.58_40_gc6c2479 [11923.174689] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [11924.960938] Key type lgssc registered [11926.271923] Lustre: Echo OBD driver; http://www.lustre.org/ [11934.857271] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11934.906597] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11936.550125] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11942.226408] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [11946.984181] LustreError: 331156:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11947.020109] LustreError: 331156:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [11950.556844] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11952.114953] LustreError: 331155:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11957.228991] LustreError: 331156:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11958.441527] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11958.878039] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11963.880329] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [11974.949673] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11975.377954] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11981.546769] LustreError: 332835:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11981.574953] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [11983.086713] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [11990.022135] LustreError: 332838:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11990.036806] LustreError: 332838:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [11992.894576] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12000.242040] LustreError: 332837:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12000.254178] LustreError: 332837:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [12003.775624] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12004.161850] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12009.486796] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [12013.087702] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [12021.747556] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12031.325884] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [12031.614350] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12137.441410] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12137.446300] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12137.467299] LustreError: Skipped 1 previous similar message [12137.468041] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12137.503774] Lustre: Skipped 1 previous similar message [12142.564661] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12142.568749] Lustre: Skipped 1 previous similar message [12142.849621] Lustre: server umount lustre-OST0001 complete [12147.685401] LustreError: 333129:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12147.714816] LustreError: 333129:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [12150.758957] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12150.768059] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12150.780238] Lustre: Skipped 1 previous similar message [12150.789360] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12152.808138] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12152.816789] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12157.923331] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12157.945164] Lustre: Skipped 2 previous similar messages [12164.065494] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12164.373949] Lustre: server umount lustre-OST0000 complete [12173.282904] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12173.302644] Lustre: Skipped 1 previous similar message [12173.319910] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12173.331641] Lustre: Skipped 3 previous similar messages [12175.317601] Lustre: server umount lustre-MDT0000 complete [12179.941647] LustreError: 331135:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789460083 with bad export cookie 13823163757770674161 [12179.954965] LustreError: 331135:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12179.961600] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12180.528513] Lustre: server umount lustre-MDT0001 complete [12194.955816] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [12198.639962] Key type lgssc unregistered [12199.064061] LNet: 336101:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12199.075438] LNetError: 336101:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12199.104053] LNet: Removed LNI 192.168.201.124@tcp [12200.338565] Key type .llcrypt unregistered [12200.342312] Key type ._llcrypt unregistered [12249.545877] Key type ._llcrypt registered [12249.547239] Key type .llcrypt registered [12249.645574] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [12251.008654] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12251.067991] alg: No test for adler32 (adler32-zlib) [12252.203711] Lustre: Lustre: Build Version: 2.17.58_40_gc6c2479 [12252.473681] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [12254.155688] Key type lgssc registered [12255.182550] Lustre: Echo OBD driver; http://www.lustre.org/ [12264.968381] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12264.993040] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12266.720671] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12272.549550] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [12276.198348] LustreError: 338390:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12276.215742] LustreError: 338390:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [12281.321910] LustreError: 338391:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12283.224136] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12283.741801] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12290.032935] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [12293.730384] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [12302.093790] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12302.457797] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12309.623341] LustreError: 339882:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12310.493504] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [12311.670260] LustreError: 339881:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12311.677932] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [12317.162145] LustreError: 340193:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12317.200984] LustreError: 340193:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [12321.687524] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12321.903265] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12327.444295] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [12330.499676] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [12341.549571] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12350.816612] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [12363.242644] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12363.256244] Lustre: Skipped 3 previous similar messages [12363.256862] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12363.271142] Lustre: Skipped 3 previous similar messages [12368.357705] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12368.365217] Lustre: Skipped 3 previous similar messages [12368.772247] Lustre: server umount lustre-MDT0000 complete [12372.537031] LustreError: 338371:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789460276 with bad export cookie 1843830139146993254 [12372.549144] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12373.133608] Lustre: server umount lustre-MDT0001 complete [12386.432218] Lustre: server umount lustre-OST0000 complete [12389.665839] Lustre: 337654:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789460277/real 1789460277] req@ffff8b803ee9d500 x1876384972605952/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789460293 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [12389.691942] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12391.132753] Lustre: server umount lustre-OST0001 complete [12411.377687] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [12422.407335] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12422.787085] LustreError: 343657:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12422.867048] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12427.967239] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [12437.339205] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12437.871235] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12444.298071] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [12448.303954] Lustre: 344797:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [12456.543618] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12457.067789] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12462.194506] LustreError: 345151:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12462.214800] LustreError: 345151:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [12464.228606] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [12467.324528] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [12473.364125] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12478.973631] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [12479.664658] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [12488.419990] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12492.676647] Lustre: 346669:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [12499.433121] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12499.444763] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12499.462791] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12499.474324] Lustre: Skipped 3 previous similar messages [12503.364591] Lustre: server umount lustre-MDT0000 complete [12504.551405] LustreError: 343653:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12504.581724] LustreError: 343653:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 6 previous similar messages [12508.106179] LustreError: 343640:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789460411 with bad export cookie 1843830139146994955 [12508.120618] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12508.131393] LustreError: 343640:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [12508.732697] Lustre: server umount lustre-MDT0001 complete [12525.879500] Lustre: server umount lustre-OST0000 complete [12526.048294] Lustre: 337651:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789460413/real 1789460413] req@ffff8b802f10a300 x1876384972655616/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789460429 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [12526.108729] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12526.139354] Lustre: Skipped 3 previous similar messages [12529.890962] Lustre: 337654:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789460417/real 1789460417] req@ffff8b803ba70e00 x1876384972655872/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789460433 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [12532.239647] Lustre: server umount lustre-OST0001 complete [12537.131376] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [12539.238486] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [12541.158761] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 04:20:43 (1789460443) [12564.215168] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [12572.457811] loop0: detected capacity change from 0 to 1099511627776 [12573.367760] SGI XFS with ACLs, security attributes, quota, no debug enabled [12573.373192] *** VALIDATE xfs *** [12573.381739] XFS (loop0): Mounting V5 Filesystem [12573.405833] XFS (loop0): Ending clean mount [12574.968442] loop1: detected capacity change from 0 to 18691697672192 [12691.108337] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [12703.964888] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [12748.719561] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12749.027148] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12749.119360] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12749.188583] Lustre: lustre-MDT0000: new disk, initializing [12749.260870] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12749.265632] Lustre: Skipped 1 previous similar message [12749.281621] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12754.543953] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [12765.476482] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12765.583065] Lustre: 351810:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12765.623444] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12765.625434] Lustre: Skipped 1 previous similar message [12765.687423] Lustre: lustre-MDT0001: new disk, initializing [12765.779830] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12765.785179] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12770.791135] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [12782.400739] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12788.989917] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12797.459209] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12797.805841] Lustre: lustre-OST0000: new disk, initializing [12797.816350] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12797.825234] Lustre: 353778:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12799.687709] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12799.704340] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12799.845423] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12805.634670] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [12815.626107] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12821.130520] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12821.319559] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12826.075482] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [12826.391407] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12830.695904] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12830.699601] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12830.722785] LustreError: Skipped 1 previous similar message [12830.724507] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12830.736339] Lustre: Skipped 1 previous similar message [12834.674364] Lustre: server umount lustre-OST0000 complete [12840.933681] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12840.939844] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12840.949880] Lustre: Skipped 2 previous similar messages [12840.962681] Lustre: Skipped 1 previous similar message [12845.289750] Lustre: server umount lustre-MDT0000 complete [12846.051133] LustreError: 351821:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12846.102342] LustreError: 351821:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [12849.108940] LustreError: 353777:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789460752 with bad export cookie 1843830139146996509 [12849.114555] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12849.119855] LustreError: 353777:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12849.496371] Lustre: server umount lustre-MDT0001 complete [12857.934448] XFS (loop0): Unmounting Filesystem [12862.976929] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 04:26:04 (1789460764) [12870.706438] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12871.231511] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12871.240345] Lustre: Skipped 2 previous similar messages [12875.417320] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [12885.821683] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12891.156091] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [12901.658551] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12908.398731] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12917.250982] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12926.714310] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [12934.122160] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12956.653056] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12956.653395] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12956.659105] LustreError: Skipped 1 previous similar message [12956.662397] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12956.669358] Lustre: Skipped 1 previous similar message [12956.680114] Lustre: Skipped 1 previous similar message [12961.328214] Lustre: server umount lustre-OST0000 complete [12971.943874] Lustre: server umount lustre-MDT0000 complete [12976.514670] LustreError: 356321:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789460880 with bad export cookie 1843830139146997384 [12976.527479] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12976.823664] Lustre: server umount lustre-MDT0001 complete [12987.889213] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [12990.137967] Key type lgssc unregistered [12990.401716] LNet: 359990:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12990.415330] LNetError: 359990:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12990.434213] LNet: Removed LNI 192.168.201.124@tcp [12991.344636] Key type .llcrypt unregistered [12991.346571] Key type ._llcrypt unregistered [13019.870359] Key type ._llcrypt registered [13019.872371] Key type .llcrypt registered [13019.981132] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [13020.926847] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [13021.021067] alg: No test for adler32 (adler32-zlib) [13022.367841] Lustre: Lustre: Build Version: 2.17.58_40_gc6c2479 [13022.792988] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [13024.528490] Key type lgssc registered [13025.827654] Lustre: Echo OBD driver; http://www.lustre.org/ [13035.479285] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [13035.510311] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13036.985473] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13041.769573] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [13046.246164] LustreError: 361554:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13046.281988] LustreError: 361554:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [13051.322753] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13051.377130] LustreError: 361555:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13051.872753] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13058.647855] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [13066.386146] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13072.181790] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13078.269299] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13078.564432] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13084.293786] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [13092.538349] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13103.074886] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13103.077881] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13103.106219] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13103.115292] Lustre: Skipped 1 previous similar message [13104.461457] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.24@tcp (stopping) [13108.201206] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13108.207718] Lustre: Skipped 1 previous similar message [13108.691955] Lustre: server umount lustre-MDT0001 complete [13109.518932] LustreError: 361551:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.24@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13113.319887] LustreError: 362475:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13113.345975] LustreError: 362475:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [13118.437781] LustreError: 361550:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13118.467185] LustreError: 361550:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [13121.106395] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13121.307177] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13121.312143] Lustre: Found index 1 for lustre-MDT0001, updating log [13121.389777] Lustre: 361543:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13121.746845] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13126.929229] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [13127.155542] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13128.211994] LustreError: 365065:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13128.223125] LustreError: 365065:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13128.231785] LustreError: 361546:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13128.301253] LustreError: 365074:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.124@tcp: cfg command failed: rc = -17 [13128.318130] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [13128.318130] [13140.270337] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.24@tcp (stopping) [13142.501779] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13142.505077] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13142.537626] Lustre: Skipped 1 previous similar message [13145.362928] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.24@tcp (stopping) [13145.365952] Lustre: Skipped 2 previous similar messages [13145.899399] Lustre: server umount lustre-MDT0001 complete [13147.619783] LustreError: 361551:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13147.630622] LustreError: 361551:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [13157.807246] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13157.935766] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13157.942556] Lustre: Found index 1 for lustre-MDT0001, updating log [13157.980336] Lustre: 361544:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13158.372700] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13163.051229] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [13163.519284] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13163.533342] LustreError: 366097:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13163.539865] LustreError: 366097:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13163.546298] LustreError: 361546:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13163.553541] LustreError: 361546:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13163.658875] LustreError: 366100:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [13163.668599] LustreError: 366100:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [13163.672328] LustreError: 366100:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.124@tcp: cfg command failed: rc = -17 [13163.686607] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.201.124@tcp [13163.686607] [13470.691549] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13470.695411] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13470.713057] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13470.724684] Lustre: Skipped 2 previous similar messages [13470.743182] Lustre: Skipped 1 previous similar message [13474.931263] Lustre: server umount lustre-MDT0001 complete [13475.823844] LustreError: 370927:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13475.851040] LustreError: 370927:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 7 previous similar messages [13485.667043] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13485.780309] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13485.783086] Lustre: Found index 1 for lustre-MDT0001, updating log [13485.829159] Lustre: 361543:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13486.174776] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13490.994182] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [13491.181108] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13491.707286] LustreError: 374235:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13491.728806] LustreError: 374235:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13491.753021] LustreError: 361546:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13491.761640] LustreError: 361546:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13491.857617] LustreError: 374245:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [13491.876292] LustreError: 374245:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.124@tcp: cfg command failed: rc = -114 [13491.884120] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [13491.884120] [13818.854246] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13818.878159] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13818.893708] Lustre: Skipped 1 previous similar message [13818.896130] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13818.930279] Lustre: Skipped 3 previous similar messages [13820.374202] Lustre: server umount lustre-MDT0000 complete [13823.972378] LustreError: 361551:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13824.006804] LustreError: 361551:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 6 previous similar messages [13824.664353] LustreError: 364738:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789461728 with bad export cookie 1274647611289391343 [13824.676429] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13829.106671] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13829.124165] Lustre: Skipped 1 previous similar message [13831.379740] Lustre: server umount lustre-MDT0001 complete [13836.928458] Lustre: server umount lustre-OST0000 complete [13849.487638] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [13863.349954] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [13901.070459] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 04:43:22 (1789461802) [13907.874034] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13908.141933] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13908.166908] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13908.266860] Lustre: lustre-MDT0000: new disk, initializing [13908.340766] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13908.353868] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13913.697803] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [13926.038708] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13926.203392] Lustre: 384889:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13926.300978] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13926.306982] Lustre: Skipped 1 previous similar message [13926.443938] Lustre: lustre-MDT0001: new disk, initializing [13926.573224] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13926.622980] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13926.642081] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13932.801884] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [13942.849682] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13948.944803] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13956.563872] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13956.844862] Lustre: lustre-OST0000: new disk, initializing [13956.849876] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13956.858220] Lustre: 386858:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13956.946334] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13965.117874] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [13966.449171] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13966.461451] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13966.711452] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13978.508659] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13988.322721] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13988.328246] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13988.349027] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13988.360502] Lustre: Skipped 1 previous similar message [13994.617652] Lustre: server umount lustre-MDT0000 complete [13997.544786] LustreError: 384899:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13997.564974] LustreError: 384899:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [13998.963281] LustreError: 384881:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789461902 with bad export cookie 1274647611289393296 [13998.967664] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13998.972895] LustreError: 384881:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [13999.669746] Lustre: server umount lustre-MDT0001 complete [14014.436870] Lustre: server umount lustre-OST0000 complete [14047.260522] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [14058.044833] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [14090.716309] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 13787 sec ========= 04:46:32 (1789461992) [14092.399127] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 04:46:34 (1789461994) === [14096.547261] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 04:46:38 (1789461998) === [14145.575249] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [14148.322140] Key type lgssc unregistered [14148.588296] LNet: 393225:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [14148.602422] LNetError: 393225:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [14148.625594] LNet: Removed LNI 192.168.201.124@tcp [14149.501597] Key type .llcrypt unregistered [14149.503741] Key type ._llcrypt unregistered