[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 479232838 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001016] APIC: Switch to symmetric I/O mode setup [ 0.003339] x2apic enabled [ 0.004014] Switched APIC routing to physical x2apic. [ 0.005023] kvm-guest: setup PV IPIs [ 0.008000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008033] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009037] pid_max: default: 32768 minimum: 301 [ 0.010208] LSM: Security Framework initializing [ 0.011078] Yama: becoming mindful. [ 0.012062] SELinux: Initializing. [ 0.013114] *** VALIDATE selinux *** [ 0.021283] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.025825] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028111] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030121] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.032149] *** VALIDATE tmpfs *** [ 0.034052] *** VALIDATE proc *** [ 0.035339] *** VALIDATE cgroup *** [ 0.036019] *** VALIDATE cgroup2 *** [ 0.037328] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.039016] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.040016] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.041043] Spectre V2 : User space: Vulnerable [ 0.042015] Speculative Store Bypass: Vulnerable [ 0.045155] debug: unmapping init [mem 0xffffffff86e59000-0xffffffff86e60fff] [ 0.048223] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.049815] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.050035] ... version: 2 [ 0.051020] ... bit width: 48 [ 0.052019] ... generic registers: 4 [ 0.053022] ... value mask: 0000ffffffffffff [ 0.054018] ... max period: 00007fffffffffff [ 0.055019] ... fixed-purpose events: 3 [ 0.056016] ... event mask: 000000070000000f [ 0.057393] rcu: Hierarchical SRCU implementation. [ 0.059599] smp: Bringing up secondary CPUs ... [ 0.060673] x86: Booting SMP configuration: [ 0.061039] .... node #0, CPUs: #1 #2 #3 [ 0.064499] smp: Brought up 1 node, 4 CPUs [ 0.066021] smpboot: Max logical packages: 1 [ 0.067031] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.248196] node 0 deferred pages initialised in 179ms [ 0.252187] devtmpfs: initialized [ 0.253325] x86/mm: Memory block size: 128MB [ 0.256198] gcov: version magic: 0x41383552 [ 0.258250] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.262111] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.265329] pinctrl core: initialized pinctrl subsystem [ 0.267206] [ 0.267632] ************************************************************* [ 0.269016] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.271013] ** ** [ 0.273017] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.276018] ** ** [ 0.278014] ** This means that this kernel is built to expose internal ** [ 0.279013] ** IOMMU data structures, which may compromise security on ** [ 0.281016] ** your system. ** [ 0.282015] ** ** [ 0.285015] ** If you see this message and you are not debugging the ** [ 0.287013] ** kernel, report this immediately to your vendor! ** [ 0.289014] ** ** [ 0.291014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.294016] ************************************************************* [ 0.296774] NET: Registered protocol family 16 [ 0.298483] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.301067] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.302091] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.306177] cpuidle: using governor menu [ 0.307552] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.310777] PCI: Using configuration type 1 for base access [ 0.312160] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.323127] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.324028] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.326083] cryptd: max_cpu_qlen set to 1000 [ 0.330376] ACPI: Added _OSI(Module Device) [ 0.332020] ACPI: Added _OSI(Processor Device) [ 0.334023] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.336019] ACPI: Added _OSI(Processor Aggregator Device) [ 0.340812] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.347561] ACPI: Interpreter enabled [ 0.349081] ACPI: PM: (supports S0 S3 S4 S5) [ 0.351023] ACPI: Using IOAPIC for interrupt routing [ 0.353114] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.356278] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.365000] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.367047] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.369016] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.371113] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.376362] acpiphp: Slot [2] registered [ 0.377120] acpiphp: Slot [5] registered [ 0.378083] acpiphp: Slot [6] registered [ 0.380197] acpiphp: Slot [7] registered [ 0.381131] acpiphp: Slot [8] registered [ 0.383165] acpiphp: Slot [9] registered [ 0.386159] acpiphp: Slot [10] registered [ 0.387151] acpiphp: Slot [3] registered [ 0.389118] acpiphp: Slot [4] registered [ 0.390120] acpiphp: Slot [11] registered [ 0.392102] acpiphp: Slot [12] registered [ 0.393100] acpiphp: Slot [13] registered [ 0.394096] acpiphp: Slot [14] registered [ 0.396108] acpiphp: Slot [15] registered [ 0.397145] acpiphp: Slot [16] registered [ 0.399110] acpiphp: Slot [17] registered [ 0.401127] acpiphp: Slot [18] registered [ 0.402125] acpiphp: Slot [19] registered [ 0.403109] acpiphp: Slot [20] registered [ 0.405131] acpiphp: Slot [21] registered [ 0.406141] acpiphp: Slot [22] registered [ 0.408119] acpiphp: Slot [23] registered [ 0.409130] acpiphp: Slot [24] registered [ 0.411106] acpiphp: Slot [25] registered [ 0.412121] acpiphp: Slot [26] registered [ 0.413145] acpiphp: Slot [27] registered [ 0.415104] acpiphp: Slot [28] registered [ 0.417136] acpiphp: Slot [29] registered [ 0.418140] acpiphp: Slot [30] registered [ 0.420146] acpiphp: Slot [31] registered [ 0.422074] PCI host bridge to bus 0000:00 [ 0.423020] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.425030] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.428033] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.430027] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.433030] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.435031] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.438187] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.441012] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.444429] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.451721] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.457063] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.460021] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.462019] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.464021] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.467714] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.470939] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.473037] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.475621] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.479016] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.490953] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.495017] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.501280] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.511021] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.521018] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.541020] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.557836] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.567019] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.577020] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.593020] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.599307] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.607014] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.612022] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.628018] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.635942] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.645020] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.651020] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.669022] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.677864] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.682019] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.686016] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.702018] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.710483] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.717021] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.728038] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.748023] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.761130] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.763272] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.765279] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.766393] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.769230] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.775028] iommu: Default domain type: Passthrough [ 0.776472] SCSI subsystem initialized [ 0.777183] ACPI: bus type USB registered [ 0.778205] usbcore: registered new interface driver usbfs [ 0.780289] usbcore: registered new interface driver hub [ 0.782100] usbcore: registered new device driver usb [ 0.784196] pps_core: LinuxPPS API ver. 1 registered [ 0.786014] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.789145] PTP clock support registered [ 0.792088] EDAC MC: Ver: 3.0.0 [ 0.794123] PCI: Using ACPI for IRQ routing [ 0.795905] NetLabel: Initializing [ 0.797011] NetLabel: domain hash size = 128 [ 0.799013] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.800087] NetLabel: unlabeled traffic allowed by default [ 0.802104] vgaarb: loaded [ 0.803353] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.805024] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.812226] clocksource: Switched to clocksource kvm-clock [ 0.924560] VFS: Disk quotas dquot_6.6.0 [ 0.926677] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.929986] *** VALIDATE ramfs *** [ 0.931575] *** VALIDATE hugetlbfs *** [ 0.933344] pnp: PnP ACPI init [ 0.936751] pnp: PnP ACPI: found 6 devices [ 0.966308] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.969665] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.971992] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.974436] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.976882] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.979425] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.982362] NET: Registered protocol family 2 [ 0.985060] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.990639] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.994500] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.999424] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.002917] TCP: Hash tables configured (established 65536 bind 65536) [ 1.006001] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.009381] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.012329] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.015191] NET: Registered protocol family 1 [ 1.018786] RPC: Registered named UNIX socket transport module. [ 1.020714] RPC: Registered udp transport module. [ 1.022197] RPC: Registered tcp transport module. [ 1.023674] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.024950] NET: Registered protocol family 44 [ 1.026461] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.027856] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.029264] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.030793] PCI: CLS 0 bytes, default 64 [ 1.031966] Unpacking initramfs... [ 2.487824] debug: unmapping init [mem 0xffff9e61fcc54000-0xffff9e61fffbffff] [ 2.491606] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.493966] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.496919] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.989474] Initialise system trusted keyrings [ 2.990964] Key type blacklist registered [ 2.993080] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.003214] zbud: loaded [ 3.006822] *** VALIDATE nfs *** [ 3.008229] *** VALIDATE nfs4 *** [ 3.009980] pstore: using deflate compression [ 3.014965] Platform Keyring initialized [ 3.143765] NET: Registered protocol family 38 [ 3.145554] Key type asymmetric registered [ 3.146749] Asymmetric key parser 'x509' registered [ 3.148406] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.151770] io scheduler mq-deadline registered [ 3.153666] io scheduler kyber registered [ 3.155618] io scheduler bfq registered [ 3.158113] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.161204] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.164431] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.167898] ACPI: Power Button [PWRF] [ 3.173520] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.179103] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.190352] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.197487] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.215037] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.242065] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.269676] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.274925] Non-volatile memory driver v1.3 [ 3.276361] Linux agpgart interface v0.103 [ 3.309904] virtio_blk virtio1: [vda] 146664 512-byte logical blocks (75.1 MB/71.6 MiB) [ 3.312780] vda: detected capacity change from 0 to 75091968 [ 3.333754] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.337596] vdb: detected capacity change from 0 to 1073741824 [ 3.353394] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.355801] vdc: detected capacity change from 0 to 2621440000 [ 3.370730] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.374551] vdd: detected capacity change from 0 to 2621440000 [ 3.391494] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.395033] vde: detected capacity change from 0 to 4294967296 [ 3.412035] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.415011] vdf: detected capacity change from 0 to 4294967296 [ 3.421384] libphy: Fixed MDIO Bus: probed [ 3.430457] usbcore: registered new interface driver usbserial_generic [ 3.432664] usbserial: USB Serial support registered for generic [ 3.434356] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.437683] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.438993] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.440840] mousedev: PS/2 mouse device common for all mice [ 3.443438] rtc_cmos 00:05: RTC can wake from S4 [ 3.444384] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.445706] rtc_cmos 00:05: registered as rtc0 [ 3.449214] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.451194] intel_pstate: CPU model not supported [ 3.453208] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.455116] hid: raw HID events driver (C) Jiri Kosina [ 3.457315] usbcore: registered new interface driver usbhid [ 3.457392] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.459206] usbhid: USB HID core driver [ 3.462634] drop_monitor: Initializing network drop monitor service [ 3.464694] Initializing XFRM netlink socket [ 3.466813] NET: Registered protocol family 10 [ 3.469957] Segment Routing with IPv6 [ 3.471807] NET: Registered protocol family 17 [ 3.474407] mpls_gso: MPLS GSO support [ 3.483210] RAS: Correctable Errors collector initialized. [ 3.485267] AVX version of gcm_enc/dec engaged. [ 3.486427] AES CTR mode by8 optimization enabled [ 3.574752] sched_clock: Marking stable (3574712599, 0)->(4535920871, -961208272) [ 3.578465] registered taskstats version 1 [ 3.580189] Loading compiled-in X.509 certificates [ 3.582207] zswap: loaded using pool lzo/zbud [ 3.607377] Key type big_key registered [ 3.619350] Key type encrypted registered [ 3.621136] ima: No TPM chip found, activating TPM-bypass! [ 3.623107] ima: Allocated hash algorithm: sha1 [ 3.624521] ima: No architecture policies found [ 3.626208] evm: Initialising EVM extended attributes: [ 3.627789] evm: security.selinux [ 3.628815] evm: security.ima [ 3.629839] evm: security.capability [ 3.631208] evm: HMAC attrs: 0x1 [ 3.633681] rtc_cmos 00:05: setting system clock to 2026-09-01 11:20:47 UTC (1788261647) [ 3.639716] debug: unmapping init [mem 0xffffffff87e03000-0xffffffff87ffffff] [ 3.642137] debug: unmapping init [mem 0xffffffff86b82000-0xffffffff86e58fff] [ 3.649185] Write protecting the kernel read-only data: 28672k [ 3.651671] debug: unmapping init [mem 0xffffffff85203000-0xffffffff853fffff] [ 3.653768] debug: unmapping init [mem 0xffffffff85b14000-0xffffffff85bfffff] [ 3.684457] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.690615] systemd[1]: Detected virtualization kvm. [ 3.692154] systemd[1]: Detected architecture x86-64. [ 3.693515] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.723945] systemd[1]: No hostname configured. [ 3.725783] systemd[1]: Set hostname to . [ 3.727979] random: systemd: uninitialized urandom read (16 bytes read) [ 3.730108] systemd[1]: Initializing machine ID from random generator. [ 3.786882] random: ln: uninitialized urandom read (6 bytes read) [ 3.874734] random: systemd: uninitialized urandom read (16 bytes read) [ 3.876695] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 3.880461] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ 3.885252] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Swap. [ OK ] Reached target Timers. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Local File Systems. [ OK ] Listening on Journal Socket. Starting Setup Virtual Console... [ OK ] Reached target Sockets. Starting Create Volatile Files and Directories... Starting Create list of required st…ce nodes for the current kernel... Starting Journal Service... [ OK ] Started Memstrack Anylazing Service. Starting Apply Kernel Variables... [ OK ] Reached target Slices. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.566222] device-mapper: uevent: version 1.0.3 [ 4.568928] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.365659] random: fast init done [ 5.407727] virtio_net virtio0 ens2: renamed from eth0 [ 5.407947] scsi host0: ata_piix [ 5.439138] scsi host1: ata_piix [ 5.441203] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.443506] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.374390] dracut-initqueue[581]: RTNETLINK answers: File exists [ 10.138258] random: crng init done [ 10.139888] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.696284] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.938971] printk: systemd: 26 output lines suppressed due to ratelimiting [ 12.215911] SELinux: Disabled at runtime. [ 12.280045] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.288717] systemd[1]: Detected virtualization kvm. [ 12.290166] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.833410] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.836328] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.840805] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.845311] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.849159] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.858327] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.878448] systemd[1]: Listening on Process Core Dump Socket. [ OK ] Listening on Process Core Dump Socket. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Stopped target Switch Root. Activating swap /dev/disk/by-label/SWAP... [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Listening on udev Control Socket. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Created slice system-serial\x2dgetty.slice. Starting Remount Root and Kernel File Systems... [ 12.915243] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on udev Kernel Socket. [ OK ] Created slice system-getty.slice. [ OK ] Stopped target Initrd Root File System. Starting udev Coldplug all Devices... [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. Mounting Kernel Debug File System... [ OK ] Reached target rpc_pipefs.target. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Stopped target Initrd File Systems. Starting Apply Kernel Variables... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. Mounting POSIX Message Queue File System... Mounting Huge Pages File System... [ OK ] Reached target Paths. [ OK ] Started Journal Service. [ OK ] Activated swap /dev/disk/by-label/SWAP. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Huge Pages File System. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 13.379120] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.697380] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.743136] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.928359] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.941710] EDAC sbridge: Ver: 1.1.2 [ 15.682183] Key type dns_resolver registered [ 15.998930] NFS: Registering the id_resolver key type [ 16.001194] Key type id_resolver registered [ 16.002875] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started irqbalance daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Login Service... [ OK ] Reached target sshd-keygen.target. [ OK ] Started daily update of the root trust anchor for DNSSEC. Starting Restore /run/initramfs on shutdown... [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting System Logging Service... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg135-server login: [ 42.363363] libcfs: loading out-of-tree module taints kernel. [ 42.385373] Key type ._llcrypt registered [ 42.386521] Key type .llcrypt registered [ 42.441177] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 53.079572] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 53.940255] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 53.948057] alg: No test for adler32 (adler32-zlib) [ 55.009371] Lustre: Lustre: Build Version: 2.17.57_103_ga9ee299 [ 55.359721] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 57.000393] Key type lgssc registered [ 59.358802] Lustre: Echo OBD driver; http://www.lustre.org/ [ 77.763022] hrtimer: interrupt took 2030029 ns [ 80.731132] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 131.412624] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 150.598483] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 150.636601] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 152.040988] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 152.079213] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 152.232840] Lustre: lustre-MDT0000: new disk, initializing [ 152.334496] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 152.373300] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 158.529988] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 176.407305] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 176.508210] Lustre: 6489:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 176.547792] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 176.557192] Lustre: Skipped 1 previous similar message [ 176.688864] Lustre: lustre-MDT0001: new disk, initializing [ 176.834687] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 176.884926] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 176.901544] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 182.391916] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 188.312519] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 200.787667] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 201.201973] Lustre: lustre-OST0000: new disk, initializing [ 201.206928] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 201.227394] Lustre: 8426:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 201.388258] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 209.701953] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 211.003937] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 211.011545] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 211.131273] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 227.882614] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 228.051891] Lustre: lustre-OST0001: new disk, initializing [ 228.055813] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 228.062770] Lustre: 9502:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 228.168915] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 235.659318] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 235.671950] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 235.863820] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 238.099419] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 252.057222] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 258.923537] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 267.400323] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing check_logdir /tmp/testlogs/ [ 273.783743] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing yml_node [ 279.297638] Lustre: DEBUG MARKER: Client: 2.17.57.103 [ 283.260946] Lustre: DEBUG MARKER: MDS: 2.17.57.103 [ 286.779660] Lustre: DEBUG MARKER: OSS: 2.17.57.103 [ 288.689407] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Tue Sep 1 07:25:30 EDT 2026 [ 312.289942] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 314.573974] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 333.289945] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 333.298893] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 333.302037] Lustre: Skipped 1 previous similar message [ 333.324072] Lustre: Skipped 3 previous similar messages [ 338.406257] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 338.424696] Lustre: Skipped 3 previous similar messages [ 339.718598] Lustre: server umount lustre-MDT0000 complete [ 343.695778] LustreError: 6479:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788261987 with bad export cookie 10450408064860306715 [ 343.700633] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 343.703515] LustreError: 6479:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 348.641105] LustreError: 11066:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 348.642106] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 348.645384] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 348.661745] LustreError: 11066:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 348.691820] Lustre: Skipped 3 previous similar messages [ 350.225119] Lustre: server umount lustre-MDT0001 complete [ 355.063541] Lustre: server umount lustre-OST0000 complete [ 360.723699] Lustre: server umount lustre-OST0001 complete [ 369.174506] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 381.126858] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 422.898208] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 423.330544] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 423.384354] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 423.601943] Lustre: lustre-MDT0000: new disk, initializing [ 423.808355] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 423.859099] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 429.292610] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 441.763486] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 441.957783] Lustre: 15059:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 442.044284] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 442.049506] Lustre: Skipped 1 previous similar message [ 442.221916] Lustre: lustre-MDT0001: new disk, initializing [ 442.377192] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 442.451679] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 442.471932] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 447.813439] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 458.415680] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 464.674867] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 473.159414] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 473.836387] Lustre: lustre-OST0000: new disk, initializing [ 473.850603] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 473.876503] Lustre: 17031:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 474.080133] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 475.093261] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 475.107750] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 475.314096] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 481.709772] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 492.998239] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 497.898119] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 498.195955] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 502.647892] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 502.867439] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 505.830192] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 505.843156] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 505.869636] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 505.874822] Lustre: Skipped 1 previous similar message [ 506.365883] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 511.474906] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 511.492878] Lustre: Skipped 2 previous similar messages [ 519.136640] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 519.357684] Lustre: server umount lustre-OST0000 complete [ 524.261804] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 524.276337] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 524.300819] Lustre: Skipped 1 previous similar message [ 524.305796] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 524.307833] Lustre: Skipped 3 previous similar messages [ 529.916978] Lustre: server umount lustre-MDT0000 complete [ 531.938659] LustreError: 15071:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 531.963082] LustreError: 15071:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 534.274867] LustreError: 15868:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788262178 with bad export cookie 10450408064860308248 [ 534.276544] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 534.297123] LustreError: 15868:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 534.975524] Lustre: server umount lustre-MDT0001 complete [ 542.346658] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 07:29:43 (1788262183) [ 550.393045] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 551.003031] LustreError: 19298:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 551.190154] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 556.351865] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 556.513246] LustreError: 19298:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 556.541338] LustreError: 19298:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 560.617643] LustreError: 19299:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 566.963632] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 573.304569] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 581.247578] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 587.388967] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 596.176509] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 603.556500] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 613.153330] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 628.967660] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 638.946586] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 638.952612] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 638.963464] Lustre: Skipped 1 previous similar message [ 638.969693] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 638.972524] Lustre: Skipped 2 previous similar messages [ 642.692818] Lustre: server umount lustre-MDT0000 complete [ 646.117305] LustreError: 19298:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 646.151428] LustreError: 19298:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 647.526102] LustreError: 19279:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788262291 with bad export cookie 10450408064860309130 [ 647.528495] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 647.544746] LustreError: 19279:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 648.018879] Lustre: server umount lustre-MDT0001 complete [ 662.513682] Lustre: server umount lustre-OST0000 complete [ 688.655355] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 702.891442] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 703.422994] LustreError: 23705:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 703.529092] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 703.542525] Lustre: Skipped 2 previous similar messages [ 709.259476] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 718.639156] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 724.620032] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 727.559553] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 735.744046] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 743.063646] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 751.847154] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 751.966941] Lustre: lustre-OST0001: new disk, initializing [ 751.968858] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 751.972255] Lustre: 26023:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 753.720616] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 753.727524] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 753.837741] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 759.135697] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 769.639598] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 773.813621] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 789.995925] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 790.003239] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 790.006021] Lustre: Skipped 5 previous similar messages [ 790.012357] Lustre: Skipped 6 previous similar messages [ 793.759428] Lustre: server umount lustre-MDT0000 complete [ 795.110335] LustreError: 23700:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 795.132766] LustreError: 23700:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 797.764480] LustreError: 26993:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788262441 with bad export cookie 10450408064860309977 [ 797.769163] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 797.790990] LustreError: 26993:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 798.381965] Lustre: server umount lustre-MDT0001 complete [ 813.146702] Lustre: server umount lustre-OST0000 complete [ 816.417167] Lustre: 3619:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788262444/real 1788262444] req@ffff9e627adc2300 x1875128299556224/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788262460 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 816.453378] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 817.682710] Lustre: server umount lustre-OST0001 complete [ 837.143722] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 850.371781] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 850.742309] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 850.754127] Lustre: Skipped 3 previous similar messages [ 855.637068] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 860.200337] LustreError: 29046:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 860.242867] LustreError: 29046:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 865.479638] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 870.427761] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 873.420632] Lustre: 30194:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 880.395172] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 886.856527] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 896.620517] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 904.019389] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 912.028652] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 916.061735] Lustre: 32074:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 922.082173] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 922.090270] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 922.116600] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 928.129325] Lustre: server umount lustre-MDT0000 complete [ 931.817432] LustreError: 29027:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788262575 with bad export cookie 10450408064860311272 [ 931.823607] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 931.838860] LustreError: 29027:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 932.162558] Lustre: server umount lustre-MDT0001 complete [ 947.012956] Lustre: server umount lustre-OST0000 complete [ 949.025514] Lustre: 3618:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788262576/real 1788262576] req@ffff9e627f50f100 x1875128299603072/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788262592 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 951.069743] Lustre: server umount lustre-OST0001 complete [ 960.293055] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 07:36:42 (1788262602) [ 989.144384] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 1000.649067] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1001.214221] LustreError: 34876:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1001.229295] LustreError: 34876:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 1006.454756] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1015.354916] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1020.559676] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1023.818142] Lustre: 36025:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1031.563471] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1039.254822] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1047.322358] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1055.496800] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1064.410024] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1068.613797] Lustre: 37904:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1077.224101] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1077.235162] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1077.256360] Lustre: Skipped 4 previous similar messages [ 1077.264030] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1077.268209] Lustre: Skipped 7 previous similar messages [ 1079.483708] Lustre: server umount lustre-MDT0000 complete [ 1082.603750] LustreError: 34859:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788262726 with bad export cookie 10450408064860312378 [ 1082.605500] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1082.610483] LustreError: 34859:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1082.906354] Lustre: server umount lustre-MDT0001 complete [ 1096.277487] Lustre: server umount lustre-OST0000 complete [ 1099.233182] Lustre: 3621:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788262727/real 1788262727] req@ffff9e627adc3800 x1875128299654656/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788262743 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1100.089495] Lustre: server umount lustre-OST0001 complete [ 1109.153560] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 07:39:11 (1788262751) [ 1123.037886] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 1132.090280] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1132.614842] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1132.621578] Lustre: Skipped 7 previous similar messages [ 1135.406387] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1142.552668] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1146.480860] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1149.590526] Lustre: 41193:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1156.363959] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1163.174995] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1172.518379] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1180.284984] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1188.173990] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1191.528491] Lustre: 43070:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1192.497823] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1192.501573] Lustre: Skipped 1 previous similar message [ 1201.570982] Lustre: server umount lustre-MDT0000 complete [ 1205.067873] LustreError: 40025:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788262848 with bad export cookie 10450408064860313484 [ 1205.075742] LustreError: 40025:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1205.083317] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1208.803591] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1208.819818] Lustre: Skipped 9 previous similar messages [ 1211.718777] Lustre: server umount lustre-MDT0001 complete [ 1220.588394] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1229.281524] LustreError: 3617:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff9e627adc3480 x1875128299705472/t0(0) o250->MGC192.168.201.135@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1235.676960] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1239.173649] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1243.510633] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1247.720166] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1249.260448] Lustre: lustre-MDT0001-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 1249.264633] Lustre: Skipped 1 previous similar message [ 1252.957073] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1255.596470] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 1 sec [ 1261.981469] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1264.017646] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1269.225300] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1274.800168] Lustre: server umount lustre-MDT0000 complete [ 1274.851244] LustreError: 45072:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1274.876689] LustreError: 45072:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 34 previous similar messages [ 1278.488349] LustreError: 40027:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788262922 with bad export cookie 10450408064860314583 [ 1278.494198] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1278.503708] LustreError: 40027:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 1278.819762] Lustre: server umount lustre-MDT0001 complete [ 1293.226549] Lustre: server umount lustre-OST0000 complete [ 1295.328179] Lustre: 3618:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788262923/real 1788262923] req@ffff9e614368d500 x1875128299738112/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788262939 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1298.534409] Lustre: server umount lustre-OST0001 complete [ 1310.039836] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 07:42:31 (1788262951) [ 1324.556795] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 1336.659562] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1343.215092] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1352.111225] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1356.290635] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1358.997372] Lustre: 48837:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1365.335570] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1371.553184] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1380.058899] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1386.551928] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1393.891630] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1398.520177] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1412.431519] Lustre: server umount lustre-MDT0000 complete [ 1416.380513] LustreError: 47663:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788263060 with bad export cookie 10450408064860315402 [ 1416.390167] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1416.390176] LustreError: 47663:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1416.871492] Lustre: server umount lustre-MDT0001 complete [ 1430.856917] Lustre: server umount lustre-OST0000 complete [ 1443.901546] Lustre: server umount lustre-OST0001 complete [ 1453.264229] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 07:44:54 (1788263094) [ 1455.572609] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1457.209816] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 07:44:59 (1788263099) [ 1467.899886] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 1479.193414] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 1503.285570] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1503.582645] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1503.646819] Lustre: lustre-MDT0000: new disk, initializing [ 1503.730688] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1508.029363] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1517.045521] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1523.811256] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1524.067935] Lustre: lustre-OST0000: new disk, initializing [ 1524.071446] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1524.074491] Lustre: Skipped 1 previous similar message [ 1524.082944] Lustre: 54913:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1525.325347] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1525.334596] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1525.366181] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1530.385565] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1539.906619] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1542.424947] Lustre: DEBUG MARKER: create test files [ 1571.301400] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1571.308897] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1571.322308] Lustre: Skipped 9 previous similar messages [ 1571.329071] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1571.339907] Lustre: Skipped 19 previous similar messages [ 1574.890545] Lustre: server umount lustre-OST0000 complete [ 1578.812758] Lustre: server umount lustre-MDT0000 complete [ 1588.641580] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 1592.032767] Key type lgssc unregistered [ 1592.459902] LNet: 56542:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1592.475842] LNetError: 56542:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1592.501074] LNet: Removed LNI 192.168.201.135@tcp [ 1593.595176] Key type .llcrypt unregistered [ 1593.597762] Key type ._llcrypt unregistered [ 1623.350232] Key type ._llcrypt registered [ 1623.352405] Key type .llcrypt registered [ 1623.429412] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 1624.241124] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1624.342030] alg: No test for adler32 (adler32-zlib) [ 1625.313260] Lustre: Lustre: Build Version: 2.17.57_103_ga9ee299 [ 1625.520862] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 1627.176839] Key type lgssc registered [ 1628.425850] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1637.431549] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1637.481486] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1639.031903] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1643.991805] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1651.612671] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1658.270550] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1658.507781] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1663.675775] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:101 to 0x240000400:129) [ 1663.816588] Lustre: 59278:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1663.824577] Lustre: 59278:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1663.829367] Lustre: 59278:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1663.836596] Lustre: 59278:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1663.839841] Lustre: 59278:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1663.842793] Lustre: 59278:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1664.625467] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1672.054334] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1677.108898] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1682.275927] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1689.572306] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1689.585548] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1689.613658] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1691.577164] Lustre: server umount lustre-OST0000 complete [ 1695.843247] Lustre: server umount lustre-MDT0000 complete [ 1708.056551] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 1710.815213] Key type lgssc unregistered [ 1711.057881] LNet: 60887:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1711.068158] LNetError: 60887:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1711.092812] LNet: Removed LNI 192.168.201.135@tcp [ 1712.119748] Key type .llcrypt unregistered [ 1712.124653] Key type ._llcrypt unregistered [ 1741.308518] Key type ._llcrypt registered [ 1741.314535] Key type .llcrypt registered [ 1741.442940] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 1742.871893] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1743.028217] alg: No test for adler32 (adler32-zlib) [ 1744.322790] Lustre: Lustre: Build Version: 2.17.57_103_ga9ee299 [ 1744.719490] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 1746.473150] Key type lgssc registered [ 1747.526516] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1756.111513] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1756.142523] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1757.634370] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1761.532912] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1767.800389] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1777.071253] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1777.710657] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1782.845092] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1783.048976] Lustre: 63701:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1783.075031] Lustre: 63701:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1783.092579] Lustre: 63701:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1783.107698] Lustre: 63701:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1783.124239] Lustre: 63701:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1783.148506] Lustre: 63701:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1786.338871] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1796.083695] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1803.570744] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1818.598962] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1818.599649] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1818.618621] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1820.390491] Lustre: server umount lustre-OST0000 complete [ 1825.257412] Lustre: server umount lustre-MDT0000 complete [ 1838.814632] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 1842.334765] Key type lgssc unregistered [ 1842.705966] LNet: 65257:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1842.712824] LNetError: 65257:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1842.727757] LNet: Removed LNI 192.168.201.135@tcp [ 1843.727564] Key type .llcrypt unregistered [ 1843.729930] Key type ._llcrypt unregistered [ 1862.732342] Key type ._llcrypt registered [ 1862.733848] Key type .llcrypt registered [ 1862.828158] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 1877.837635] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 1878.891104] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1879.244657] alg: No test for adler32 (adler32-zlib) [ 1880.331886] Lustre: Lustre: Build Version: 2.17.57_103_ga9ee299 [ 1880.724816] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 1882.488139] Key type lgssc registered [ 1884.439535] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1926.273788] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1926.307403] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1927.623056] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1927.651682] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1927.719481] Lustre: lustre-MDT0000: new disk, initializing [ 1927.773809] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1927.785433] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1932.855654] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1944.376494] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1944.481927] Lustre: 68409:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1944.586048] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1944.591592] Lustre: Skipped 1 previous similar message [ 1944.673395] Lustre: lustre-MDT0001: new disk, initializing [ 1944.796076] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1944.847988] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1944.865629] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1950.070349] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1960.361380] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1967.378134] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1974.798292] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1975.244807] Lustre: lustre-OST0000: new disk, initializing [ 1975.249453] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1975.258706] Lustre: 70379:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1975.452092] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1982.710566] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1985.106155] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1985.125569] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1985.224640] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1993.760781] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1999.154639] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1999.484582] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2005.060225] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2005.353144] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2011.109376] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2011.123132] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2011.127084] LustreError: Skipped 1 previous similar message [ 2011.154048] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2011.160719] Lustre: Skipped 1 previous similar message [ 2011.185900] Lustre: Skipped 1 previous similar message [ 2013.718145] Lustre: server umount lustre-OST0000 complete [ 2021.346403] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2021.354226] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2021.356509] Lustre: Skipped 1 previous similar message [ 2021.372256] Lustre: Skipped 1 previous similar message [ 2024.615938] Lustre: server umount lustre-MDT0000 complete [ 2029.004972] LustreError: 68403:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788263672 with bad export cookie 5221655436999468596 [ 2029.012344] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2029.025304] LustreError: 68403:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2029.729358] Lustre: server umount lustre-MDT0001 complete [ 2040.470464] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 07:54:42 (1788263682) [ 2054.597607] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 2099.324152] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 2112.247505] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 2152.907046] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2153.253337] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2153.308570] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2153.424917] Lustre: lustre-MDT0000: new disk, initializing [ 2153.568870] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2153.585826] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2158.842580] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2170.238860] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2170.348568] Lustre: 76399:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2170.383831] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2170.387141] Lustre: Skipped 1 previous similar message [ 2170.450479] Lustre: lustre-MDT0001: new disk, initializing [ 2170.515691] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2170.551699] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2170.562563] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2175.913369] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2185.589360] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2191.608738] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2199.746442] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2200.137576] Lustre: lustre-OST0000: new disk, initializing [ 2200.140460] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2200.158095] Lustre: 78369:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2200.254411] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2206.812541] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2206.819049] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2207.030339] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2207.558871] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2218.674779] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2223.785191] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2224.034285] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2228.822218] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2229.021376] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2232.821457] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2232.834097] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2232.847555] Lustre: Skipped 1 previous similar message [ 2232.856845] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2237.279826] Lustre: server umount lustre-OST0000 complete [ 2242.529979] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2242.534209] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2242.552310] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2242.561902] Lustre: Skipped 1 previous similar message [ 2248.166672] LustreError: 76405:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2248.186577] LustreError: 76405:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 2248.276876] Lustre: server umount lustre-MDT0000 complete [ 2252.258389] LustreError: 76410:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2252.281722] LustreError: 76410:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2252.504981] LustreError: 78368:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788263896 with bad export cookie 5221655436999469478 [ 2252.514356] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2252.532322] LustreError: 78368:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2252.987432] Lustre: server umount lustre-MDT0001 complete [ 2261.868063] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 07:58:23 (1788263903) [ 2270.362781] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2270.930250] LustreError: 80783:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2271.012337] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2276.352081] LustreError: 80784:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2276.969185] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2280.424261] LustreError: 80783:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2287.735479] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2293.353466] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2300.897500] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2305.294951] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2312.177593] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2312.451361] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2312.461627] Lustre: Skipped 1 previous similar message [ 2318.143406] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2327.380463] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2332.248837] Lustre: 83457:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting 210ea8d1-f891-4aed-a662-6875d6e52688 at adminstrative request [ 2336.245281] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2336.268065] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2346.471054] Lustre: MGS: Client 210ea8d1-f891-4aed-a662-6875d6e52688 (at 0@lo) reconnecting [ 2346.477943] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x48770bf455d72d18 to 0x48770bf455d72fb8 [ 2346.486926] Lustre: MGC192.168.201.135@tcp: Connection restored to 0@lo (at 0@lo) [ 2369.996671] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2370.320736] Lustre: lustre-OST0001: new disk, initializing [ 2370.325802] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2370.334897] Lustre: 83849:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2377.383192] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2377.394493] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2377.516335] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2377.718365] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2388.889597] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2393.075100] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2393.080422] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2393.088017] LustreError: Skipped 1 previous similar message [ 2393.088960] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2393.088966] Lustre: Skipped 2 previous similar messages [ 2393.135861] Lustre: Skipped 2 previous similar messages [ 2398.555488] Lustre: server umount lustre-OST0001 complete [ 2402.787565] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2402.798820] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2402.822771] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2402.834855] Lustre: Skipped 3 previous similar messages [ 2403.311561] LustreError: 82606:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2403.336409] LustreError: 82606:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 2408.440782] Lustre: server umount lustre-OST0000 complete [ 2418.973093] Lustre: server umount lustre-MDT0000 complete [ 2423.588688] LustreError: 81524:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788264067 with bad export cookie 5221655436999471032 [ 2423.602064] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2423.605108] LustreError: 81524:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2424.023252] Lustre: server umount lustre-MDT0001 complete [ 2434.525591] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 08:01:15 (1788264075) [ 2436.526841] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2438.369559] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 08:01:20 (1788264080) [ 2440.446422] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2442.332198] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 08:01:24 (1788264084) [ 2444.169542] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2446.189296] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 08:01:27 (1788264087) [ 2447.949728] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2450.288427] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 08:01:31 (1788264091) [ 2452.246448] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2459.136428] loop: module loaded [ 2459.168517] loop0: detected capacity change from 0 to 4294967296 [ 2459.260582] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2460.954534] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2464.489904] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2469.294207] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2499.464222] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2499.534254] LustreError: 87984:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2499.540533] LustreError: 87984:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2499.548847] LustreError: 87984:0:(tgt_mount.c:2582:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2499.564546] LustreError: 87984:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2520.478526] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 2530.377365] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 2574.877189] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2575.105920] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2575.122898] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2575.163992] Lustre: lustre-MDT0000: new disk, initializing [ 2575.222594] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2575.233946] Lustre: Skipped 1 previous similar message [ 2575.246985] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2578.210857] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2587.722997] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2587.849620] Lustre: 90851:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2588.122718] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2593.435451] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2603.387963] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2610.522722] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2618.550618] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2618.825405] Lustre: 92819:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2627.171364] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2628.773163] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2628.947526] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2637.376868] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2643.956843] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2644.199965] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2650.049371] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2650.411772] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2654.698726] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2654.711884] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2654.717220] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2654.747339] Lustre: Skipped 4 previous similar messages [ 2654.752525] Lustre: Skipped 7 previous similar messages [ 2658.928851] Lustre: server umount lustre-OST0000 complete [ 2669.401937] Lustre: server umount lustre-MDT0000 complete [ 2670.053244] LustreError: 90862:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2670.083178] LustreError: 90862:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2673.277565] LustreError: 90843:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788264317 with bad export cookie 5221655436999471648 [ 2673.287138] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2673.293914] LustreError: 90843:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2673.741298] Lustre: server umount lustre-MDT0001 complete [ 2683.141625] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 08:05:24 (1788264324) [ 2690.521631] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2696.519648] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2706.004995] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2706.501283] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2706.508093] Lustre: Skipped 3 previous similar messages [ 2711.946694] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2720.369527] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2726.855409] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2734.424960] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2741.689831] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2750.513102] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2755.432875] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2755.737845] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2761.368636] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2761.716221] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2769.911787] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2770.047205] Lustre: lustre-OST0001: new disk, initializing [ 2770.050477] Lustre: Skipped 2 previous similar messages [ 2770.054622] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2770.059780] Lustre: Skipped 3 previous similar messages [ 2770.066343] Lustre: 98603:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2775.602945] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2775.610687] Lustre: Skipped 2 previous similar messages [ 2775.615608] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2775.709788] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2777.325952] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2788.240978] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2794.022083] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2794.270988] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2799.919568] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2800.167363] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2822.711274] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2847.431377] Lustre: Failing over lustre-MDT0000 [ 2847.714339] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2847.736810] Lustre: Skipped 4 previous similar messages [ 2847.746081] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2847.753436] Lustre: Skipped 4 previous similar messages [ 2847.836976] Lustre: server umount lustre-MDT0000 complete [ 2849.260257] LustreError: 95228:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.201.35@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2849.310252] LustreError: 95228:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 2868.199050] Lustre: 66630:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788264496/real 1788264496] req@0000000026971ab3 x1875130213223808/t0(0) o400->MGC192.168.201.135@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1788264512 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2868.237942] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2876.561314] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2876.567986] LDISKFS-fs (dm-0): recovery complete [ 2876.604412] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2878.816296] LustreError: 101425:0:(import.c:339:ptlrpc_invalidate_import()) MGS: timeout waiting for callback (1 != 0) [ 2878.828990] LustreError: 101425:0:(import.c:363:ptlrpc_invalidate_import()) @@@ still on sending list req@000000001330e216 x1875130213229184/t0(0) o250->MGC192.168.201.135@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 1788264522 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2878.879162] LustreError: 101425:0:(import.c:373:ptlrpc_invalidate_import()) MGS: Unregistering RPCs found (0). Network is sluggish? Waiting for them to error out. [ 2879.830693] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2879.877150] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2879.878170] Lustre: 95229:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2885.093600] Lustre: 95961:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2885.108679] Lustre: 95961:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 2885.130735] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 2885.233831] LustreError: 101457:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2885.561190] LustreError: 101457:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2885.695550] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2885.913934] LustreError: 101457:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2885.928661] LustreError: 101457:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2886.272767] LustreError: 101457:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2886.299604] LustreError: 101457:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2886.318970] Lustre: 101457:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2886.332505] Lustre: 101457:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 2887.022219] LustreError: 101457:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2887.033936] LustreError: 101457:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2887.361410] LustreError: 101457:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2887.375400] LustreError: 101457:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2888.357071] Lustre: 101457:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2888.377540] Lustre: 101457:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 5 previous similar messages [ 2889.025335] LustreError: 101457:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2889.035224] LustreError: 101457:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 5 previous similar messages [ 2889.664350] LustreError: 101457:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2889.669520] LustreError: 101457:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2892.533842] Lustre: 101457:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2892.548644] Lustre: 101457:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 2893.194537] LustreError: 101457:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2893.219502] LustreError: 101457:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2893.848362] LustreError: 101457:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2893.860241] LustreError: 101457:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2900.569929] Lustre: 101457:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2900.581053] Lustre: 101457:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 24 previous similar messages [ 2901.226234] LustreError: 101457:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2901.233023] LustreError: 101457:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2901.880136] LustreError: 101457:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2901.888118] LustreError: 101457:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2916.817118] Lustre: 101457:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2916.829940] Lustre: 101457:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 50 previous similar messages [ 2917.471673] LustreError: 101457:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2917.487867] LustreError: 101457:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2918.128305] LustreError: 101457:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2918.142228] LustreError: 101457:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2939.112156] Lustre: 101457:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2939.124694] Lustre: 101457:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2939.134468] Lustre: 101457:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client a2762406-b0e8-4cbf-9a4a-fd746be66e5d@192.168.201.35@tcp [ 2939.158896] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2939.167375] Lustre: 101457:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@000000002ce6a230 x1875130197280384/t0(8589934765) o101->a2762406-b0e8-4cbf-9a4a-fd746be66e5d@192.168.201.35@tcp:283/0 lens 592/0 e 0 to 0 dl 1788264593 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2939.196173] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2939.207737] Lustre: lustre-MDT0000: Denying connection for new client a2762406-b0e8-4cbf-9a4a-fd746be66e5d (at 192.168.201.35@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2939.247056] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2939.308253] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2939.317788] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2939.319141] Lustre: Skipped 2 previous similar messages [ 2939.388337] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:76 to 0x280000401:577) [ 2939.391077] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:44 to 0x2c0000401:545) [ 2955.233192] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2955.252861] LustreError: Skipped 2 previous similar messages [ 2955.262849] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2955.288310] Lustre: Skipped 1 previous similar message [ 2955.304049] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2958.946886] Lustre: server umount lustre-OST0000 complete [ 2961.905659] LustreError: 97054:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2961.931148] LustreError: 97054:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 41 previous similar messages [ 2968.902603] Lustre: server umount lustre-OST0001 complete [ 2979.305940] Lustre: server umount lustre-MDT0000 complete [ 2983.472756] LustreError: 95215:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788264627 with bad export cookie 5221655436999522860 [ 2983.476364] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2983.483925] LustreError: 95215:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2983.938607] Lustre: server umount lustre-MDT0001 complete [ 2993.014987] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 08:10:34 (1788264634) [ 3001.277830] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3001.283983] Lustre: *** cfs_fail_loc=197, val=0*** [ 3016.864845] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3021.984507] LustreError: 103429:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.135@tcp: failed processing log, type 4: rc = -110 [ 3027.120338] Lustre: *** cfs_fail_loc=197, val=0*** [ 3027.122217] Lustre: Skipped 65 previous similar messages [ 3047.712260] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3047.725187] Lustre: Skipped 3 previous similar messages [ 3055.814677] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3066.454988] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3069.286679] Lustre: server umount lustre-OST0000 complete [ 3077.300509] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 08:11:59 (1788264719) [ 3100.115772] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3100.905439] LustreError: 105054:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3100.940707] LustreError: 105054:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 3106.214056] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3116.294773] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3120.413968] LustreError: 105085:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3120.426657] LustreError: 105085:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3120.431700] LustreError: 105085:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 20, retries 0, failed: rc = -5 [ 3120.928491] Lustre: server umount lustre-MDT0000 complete [ 3156.848464] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 3169.304897] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 3212.150945] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3212.532699] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3212.560841] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3212.638412] Lustre: lustre-MDT0000: new disk, initializing [ 3212.715804] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3217.413914] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3228.503573] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3228.648046] Lustre: 109260:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3229.099259] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3234.532423] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3244.419514] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3250.040740] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3257.477078] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3257.756503] Lustre: 111229:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3263.262047] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3264.551064] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3274.204137] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3280.015418] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3280.349962] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3286.331439] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3286.549699] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3289.057242] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3289.061665] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3289.081604] LustreError: Skipped 5 previous similar messages [ 3289.082321] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3289.082325] Lustre: Skipped 9 previous similar messages [ 3289.148334] Lustre: Skipped 6 previous similar messages [ 3294.878949] Lustre: server umount lustre-OST0000 complete [ 3305.592756] Lustre: server umount lustre-MDT0000 complete [ 3310.323297] LustreError: 109254:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788264954 with bad export cookie 5221655436999533374 [ 3310.325836] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3310.339921] LustreError: 109254:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3310.993698] Lustre: server umount lustre-MDT0001 complete [ 3321.268904] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 08:16:02 (1788264962) [ 3330.874128] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 3334.214577] Key type lgssc unregistered [ 3334.482366] LNet: 113716:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3334.488156] LNetError: 113716:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3334.501380] LNet: Removed LNI 192.168.201.135@tcp [ 3335.467705] Key type .llcrypt unregistered [ 3335.469797] Key type ._llcrypt unregistered [ 3355.671304] Key type ._llcrypt registered [ 3355.674293] Key type .llcrypt registered [ 3355.858444] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 3372.432293] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 3373.343468] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3373.377251] alg: No test for adler32 (adler32-zlib) [ 3374.489298] Lustre: Lustre: Build Version: 2.17.57_103_ga9ee299 [ 3374.772856] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 3376.512261] Key type lgssc registered [ 3378.179073] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3421.421905] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3421.456043] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3422.680433] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3422.709410] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3422.784204] Lustre: lustre-MDT0000: new disk, initializing [ 3422.853847] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3422.888783] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3427.853625] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3439.586440] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3439.830947] Lustre: lustre-OST0000: new disk, initializing [ 3439.837940] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3439.840785] Lustre: Skipped 1 previous similar message [ 3439.850928] Lustre: 117887:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3439.950051] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3446.716206] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3448.961801] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3448.975976] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3449.015310] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3458.163044] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3458.387188] Lustre: lustre-OST0001: new disk, initializing [ 3458.392380] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3458.400200] Lustre: 118753:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3458.477501] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3463.770829] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3463.786714] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3463.829580] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3465.232594] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3479.013155] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3479.023151] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3479.035902] Lustre: Skipped 1 previous similar message [ 3479.040727] Lustre: Skipped 1 previous similar message [ 3480.705877] Lustre: server umount lustre-MDT0000 complete [ 3487.512701] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3492.243722] LustreError: 118750:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788265136 with bad export cookie 2882544817664043476 [ 3492.245378] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3492.267423] LustreError: 118750:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3492.400274] Lustre: server umount lustre-OST0000 complete [ 3496.706849] Lustre: server umount lustre-OST0001 complete [ 3506.144520] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 08:19:07 (1788265147) [ 3528.522472] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 08:19:30 (1788265170) [ 3538.747641] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 3564.608800] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 08:20:06 (1788265206) [ 3607.895466] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 3619.103367] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 3661.153767] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3661.349785] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3661.389890] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3661.462287] Lustre: lustre-MDT0000: new disk, initializing [ 3661.519132] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3661.535310] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3667.718370] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3680.706773] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3680.876135] Lustre: 126505:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3680.919129] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3680.922552] Lustre: Skipped 1 previous similar message [ 3681.016677] Lustre: lustre-MDT0001: new disk, initializing [ 3681.134469] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3681.197536] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3681.231592] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3687.417203] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3697.703281] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3704.462534] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3712.890960] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3713.281790] Lustre: lustre-OST0000: new disk, initializing [ 3713.288111] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3713.295501] Lustre: 128472:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3713.418686] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3718.796069] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3718.811046] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3718.980972] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3720.651033] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3731.147368] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3736.439328] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3736.641907] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3741.258412] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3741.530726] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3744.746508] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3744.747677] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3744.758024] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3744.781687] Lustre: Skipped 1 previous similar message [ 3750.035510] Lustre: server umount lustre-OST0000 complete [ 3754.981126] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3754.991284] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3754.997402] Lustre: Skipped 1 previous similar message [ 3760.112076] LustreError: 126517:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3760.123702] LustreError: 126517:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 3760.146134] Lustre: server umount lustre-MDT0000 complete [ 3764.192423] LustreError: 126496:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788265408 with bad export cookie 2882544817664044701 [ 3764.199667] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3764.201613] LustreError: 126496:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3764.554647] Lustre: server umount lustre-MDT0001 complete [ 3776.282481] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 08:23:37 (1788265417) [ 3786.618404] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3787.549083] LustreError: 130887:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3787.577640] LustreError: 130887:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3787.684588] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3792.890521] LustreError: 130888:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3795.446690] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3797.992459] LustreError: 130888:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3803.118601] LustreError: 130887:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3805.165403] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3811.084422] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3819.673250] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3826.313860] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3834.166885] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3834.720086] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3834.727346] Lustre: Skipped 1 previous similar message [ 3841.809061] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3850.837818] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3867.123503] Lustre: *** cfs_fail_loc=159, val=0*** [ 3883.479092] Lustre: lustre-MDT0000: Client b16cea23-e5df-4112-ae5b-df5069cbd221 (at 192.168.201.35@tcp) reconnecting [ 3885.538827] Lustre: *** cfs_fail_loc=159, val=0*** [ 3885.545970] Lustre: Skipped 5 previous similar messages [ 3900.874393] Lustre: lustre-MDT0000: Client b16cea23-e5df-4112-ae5b-df5069cbd221 (at 192.168.201.35@tcp) reconnecting [ 3907.553276] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3907.561187] LustreError: Skipped 1 previous similar message [ 3907.565711] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3907.573288] Lustre: Skipped 1 previous similar message [ 3907.579034] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3908.577889] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3912.613753] Lustre: server umount lustre-OST0000 complete [ 3918.307513] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3918.322475] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3918.342702] Lustre: Skipped 1 previous similar message [ 3918.357687] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3918.360931] Lustre: Skipped 2 previous similar messages [ 3923.512294] Lustre: server umount lustre-MDT0000 complete [ 3923.945362] LustreError: 130884:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3923.980648] LustreError: 130884:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3927.709535] LustreError: 130868:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788265571 with bad export cookie 2882544817664045583 [ 3927.712780] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3927.721965] LustreError: 130868:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3928.334990] Lustre: server umount lustre-MDT0001 complete [ 3940.179834] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 3943.131045] Key type lgssc unregistered [ 3943.535791] LNet: 134684:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3943.561202] LNetError: 134684:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3944.614211] LNet: Removed LNI 192.168.201.135@tcp [ 3945.905569] Key type .llcrypt unregistered [ 3945.908543] Key type ._llcrypt unregistered [ 3975.399822] Key type ._llcrypt registered [ 3975.401337] Key type .llcrypt registered [ 3975.453903] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 3976.667524] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3976.737195] alg: No test for adler32 (adler32-zlib) [ 3978.064305] Lustre: Lustre: Build Version: 2.17.57_103_ga9ee299 [ 3978.446775] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 3980.224166] Key type lgssc registered [ 3981.818907] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3989.722881] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3989.749633] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3991.376541] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3996.383278] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4001.762868] LustreError: 136125:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4001.776115] LustreError: 136125:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4005.177952] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4005.551651] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4010.923516] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4018.530847] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4024.508537] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4032.217779] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4032.475044] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4037.624444] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4039.820938] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4048.844354] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4059.787392] Lustre: *** cfs_fail_loc=159, val=0*** [ 4076.474428] Lustre: lustre-MDT0000: Client e005057a-216f-4c71-b2d4-fe671d3d0a12 (at 192.168.201.35@tcp) reconnecting [ 4079.097932] Lustre: *** cfs_fail_loc=159, val=0*** [ 4095.427984] Lustre: lustre-MDT0001: Client e005057a-216f-4c71-b2d4-fe671d3d0a12 (at 192.168.201.35@tcp) reconnecting [ 4097.039893] Lustre: *** cfs_fail_loc=159, val=0*** [ 4097.046322] Lustre: Skipped 3 previous similar messages [ 4113.326710] Lustre: lustre-MDT0001: Client e005057a-216f-4c71-b2d4-fe671d3d0a12 (at 192.168.201.35@tcp) reconnecting [ 4117.433504] Lustre: 139165:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4119.841056] Lustre: *** cfs_fail_loc=159, val=0*** [ 4119.842975] Lustre: Skipped 4 previous similar messages [ 4136.446197] Lustre: lustre-MDT0000: Client 328427d9-b5ff-404f-962a-b02993f56982 (at 192.168.201.35@tcp) reconnecting [ 4138.805252] Lustre: *** cfs_fail_loc=159, val=0*** [ 4138.812475] Lustre: Skipped 14 previous similar messages [ 4155.344473] Lustre: lustre-MDT0000: Client 328427d9-b5ff-404f-962a-b02993f56982 (at 192.168.201.35@tcp) reconnecting [ 4157.516443] Lustre: 139455:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4165.602481] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4165.615464] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4165.639758] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4168.678471] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4168.680123] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4168.693428] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4170.116672] Lustre: server umount lustre-OST0000 complete [ 4173.800104] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4173.825682] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4173.836738] Lustre: Skipped 2 previous similar messages [ 4176.077135] Lustre: server umount lustre-MDT0000 complete [ 4178.920801] LustreError: 136126:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4178.943882] LustreError: 136126:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4179.798847] LustreError: 136105:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788265823 with bad export cookie 3607685701772149310 [ 4179.804471] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4179.807624] LustreError: 136105:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4180.179822] Lustre: server umount lustre-MDT0001 complete [ 4191.666326] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 4194.439686] Key type lgssc unregistered [ 4194.697987] LNet: 140510:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4194.713663] LNetError: 140510:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4194.739369] LNet: Removed LNI 192.168.201.135@tcp [ 4195.608084] Key type .llcrypt unregistered [ 4195.609721] Key type ._llcrypt unregistered [ 4225.733172] Key type ._llcrypt registered [ 4225.735356] Key type .llcrypt registered [ 4225.852250] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 4226.894105] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4227.111361] alg: No test for adler32 (adler32-zlib) [ 4228.312165] Lustre: Lustre: Build Version: 2.17.57_103_ga9ee299 [ 4228.678558] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 4230.424175] Key type lgssc registered [ 4231.923794] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4240.738413] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4240.769457] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4242.254940] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4247.406724] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4252.652771] LustreError: 141924:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4252.693766] LustreError: 141924:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4257.158673] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4257.436995] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4261.273962] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4268.231585] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4273.590904] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4279.931242] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4280.132931] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4282.192269] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4287.349739] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4287.875455] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4296.064994] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4303.171432] Lustre: 144506:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4306.372381] Lustre: 144555:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4307.744770] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4308.954772] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4313.909290] Lustre: 144702:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4314.991454] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4315.869492] Lustre: 144800:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4323.302622] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4323.308621] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4323.323188] Lustre: Skipped 1 previous similar message [ 4323.323830] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4326.745475] Lustre: server umount lustre-OST0000 complete [ 4333.540131] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4333.561690] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4333.567148] Lustre: Skipped 1 previous similar message [ 4333.586552] Lustre: Skipped 2 previous similar messages [ 4337.102437] Lustre: server umount lustre-MDT0000 complete [ 4338.662074] LustreError: 141918:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4338.693100] LustreError: 141918:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4341.164380] LustreError: 141906:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788265985 with bad export cookie 5971596474482759396 [ 4341.169419] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4341.178427] LustreError: 141906:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4341.561645] Lustre: server umount lustre-MDT0001 complete [ 4352.638924] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 4355.316115] Key type lgssc unregistered [ 4355.550422] LNet: 145860:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4355.555392] LNetError: 145860:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4355.579415] LNet: Removed LNI 192.168.201.135@tcp [ 4356.379620] Key type .llcrypt unregistered [ 4356.381657] Key type ._llcrypt unregistered [ 4381.715192] Key type ._llcrypt registered [ 4381.717606] Key type .llcrypt registered [ 4381.815699] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 4383.320120] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4383.356347] alg: No test for adler32 (adler32-zlib) [ 4384.416824] Lustre: Lustre: Build Version: 2.17.57_103_ga9ee299 [ 4384.644442] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 4386.336152] Key type lgssc registered [ 4387.697722] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4396.221419] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4396.247602] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4397.725730] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4402.845739] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4407.268159] LustreError: 147271:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4407.285322] LustreError: 147271:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4411.699463] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4412.074155] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4416.761195] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4424.283634] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4430.362648] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4437.592955] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4437.997317] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4443.148153] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4444.844763] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4451.971277] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4456.498021] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4461.929293] Lustre: *** cfs_fail_loc=159, val=0*** [ 4461.938791] Lustre: Skipped 2 previous similar messages [ 4478.440616] Lustre: lustre-MDT0000: Client 2794e7be-212f-4723-bc35-e36500c8ef44 (at 192.168.201.35@tcp) reconnecting [ 4484.066661] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4484.080218] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4484.081464] LustreError: Skipped 2 previous similar messages [ 4484.113802] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4485.480574] Lustre: server umount lustre-OST0000 complete [ 4488.673244] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4488.682654] LustreError: Skipped 4 previous similar messages [ 4488.690128] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4488.707292] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4492.267363] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4492.267375] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4492.288404] Lustre: Skipped 1 previous similar message [ 4494.785171] Lustre: server umount lustre-MDT0000 complete [ 4497.396573] LustreError: 147266:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4497.412630] LustreError: 147266:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4499.321284] LustreError: 147251:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788266143 with bad export cookie 15092518319022809595 [ 4499.334750] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4509.892996] Lustre: server umount lustre-MDT0001 complete [ 4524.993820] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 4527.516807] Key type lgssc unregistered [ 4527.784634] LNet: 150935:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4527.790898] LNetError: 150935:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4527.802322] LNet: Removed LNI 192.168.201.135@tcp [ 4528.584564] Key type .llcrypt unregistered [ 4528.589767] Key type ._llcrypt unregistered [ 4556.528671] Key type ._llcrypt registered [ 4556.531294] Key type .llcrypt registered [ 4556.722592] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 4558.202507] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4558.238811] alg: No test for adler32 (adler32-zlib) [ 4559.375235] Lustre: Lustre: Build Version: 2.17.57_103_ga9ee299 [ 4559.754943] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 4561.536260] Key type lgssc registered [ 4563.144593] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4571.237107] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4571.268289] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4572.819639] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4578.322644] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4582.373980] LustreError: 152347:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4582.423994] LustreError: 152347:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4587.489770] LustreError: 152348:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4590.494194] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4590.932693] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4595.602761] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4604.231708] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4611.596487] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4619.652748] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4620.054795] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4627.508101] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4635.475126] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4638.402561] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4642.630170] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4663.210638] Lustre: lustre-OST0000: evicting 9ab0fdec-e47b-464d-a6af-0007d548984d (at 192.168.201.35@tcp) by administrative request [ 4663.218210] Lustre: Skipped 1 previous similar message [ 4691.941501] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4691.951352] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4691.974036] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4693.985659] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4693.994915] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4693.997646] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4694.013103] Lustre: Skipped 1 previous similar message [ 4697.479572] Lustre: server umount lustre-OST0000 complete [ 4703.716228] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4703.733371] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4703.766369] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4709.346474] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4709.355490] Lustre: Skipped 3 previous similar messages [ 4714.467134] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4714.472121] Lustre: Skipped 1 previous similar message [ 4717.024141] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4717.320473] Lustre: server umount lustre-MDT0000 complete [ 4718.565401] LustreError: 152348:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4718.590435] LustreError: 152348:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4721.439757] LustreError: 152329:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788266365 with bad export cookie 13465520532602063190 [ 4721.449729] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4721.462318] LustreError: 152329:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4721.810234] Lustre: server umount lustre-MDT0001 complete [ 4733.853307] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 4737.052688] Key type lgssc unregistered [ 4737.437708] LNet: 156419:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4737.450244] LNetError: 156419:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4738.483219] LNet: Removed LNI 192.168.201.135@tcp [ 4739.624810] Key type .llcrypt unregistered [ 4739.629628] Key type ._llcrypt unregistered [ 4815.491783] Key type ._llcrypt registered [ 4815.495513] Key type .llcrypt registered [ 4815.675539] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 4817.276343] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4817.353608] alg: No test for adler32 (adler32-zlib) [ 4818.507615] Lustre: Lustre: Build Version: 2.17.57_103_ga9ee299 [ 4818.730972] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 4820.387040] Key type lgssc registered [ 4821.764368] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4829.242250] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4829.281165] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4830.945368] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4836.143142] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4840.417935] LustreError: 158907:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4840.433562] LustreError: 158907:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4844.638516] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4845.070398] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4850.288283] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4857.352498] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4862.582927] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4869.818656] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4870.311409] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4875.446698] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4876.103491] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4882.845776] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4889.801119] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4895.204050] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4895.206527] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4895.227439] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4896.151558] Lustre: server umount lustre-OST0000 complete [ 4900.325333] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4900.331519] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4900.336262] Lustre: Skipped 2 previous similar messages [ 4900.344927] Lustre: Skipped 2 previous similar messages [ 4905.443453] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4905.461055] Lustre: Skipped 1 previous similar message [ 4905.646244] Lustre: server umount lustre-MDT0000 complete [ 4908.588103] LustreError: 158886:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788266552 with bad export cookie 14811454963983190089 [ 4908.590057] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4908.600718] LustreError: 158886:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4908.832887] Lustre: server umount lustre-MDT0001 complete [ 4917.945753] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 4920.202322] Key type lgssc unregistered [ 4920.477976] LNet: 162445:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4920.485910] LNetError: 162445:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4920.502726] LNet: Removed LNI 192.168.201.135@tcp [ 4921.326944] Key type .llcrypt unregistered [ 4921.331475] Key type ._llcrypt unregistered [ 4945.271709] Key type ._llcrypt registered [ 4945.274290] Key type .llcrypt registered [ 4945.414274] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 4946.601204] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4946.670616] alg: No test for adler32 (adler32-zlib) [ 4947.852632] Lustre: Lustre: Build Version: 2.17.57_103_ga9ee299 [ 4948.083277] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 4949.768283] Key type lgssc registered [ 4951.023936] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4964.497809] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 08:43:26 (1788266606) [ 4980.202487] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 4988.903858] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 5017.299693] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5017.319710] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5018.519147] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5018.534564] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5018.581887] Lustre: lustre-MDT0000: new disk, initializing [ 5018.628523] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5018.641380] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5022.215909] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5029.821450] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5029.931113] Lustre: 166701:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5029.984567] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5029.986973] Lustre: Skipped 1 previous similar message [ 5030.041566] Lustre: lustre-MDT0001: new disk, initializing [ 5030.113953] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5030.144156] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5030.153561] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5033.307430] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5040.672859] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5045.340992] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5051.276682] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5051.594285] Lustre: lustre-OST0000: new disk, initializing [ 5051.598472] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5051.607303] Lustre: 168667:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5051.670594] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5056.819886] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5061.203297] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5061.212604] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5061.320052] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5064.244869] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5071.842879] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5071.845260] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5071.846760] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5071.862443] Lustre: Skipped 1 previous similar message [ 5076.963409] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5076.966016] Lustre: Skipped 1 previous similar message [ 5077.248795] Lustre: server umount lustre-OST0000 complete [ 5081.570266] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5081.589742] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5081.604457] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5081.609844] Lustre: Skipped 1 previous similar message [ 5086.645353] Lustre: server umount lustre-MDT0000 complete [ 5089.929378] LustreError: 167496:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788266733 with bad export cookie 16662463120226955024 [ 5089.932521] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5089.937114] LustreError: 167496:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5090.252586] Lustre: server umount lustre-MDT0001 complete [ 5100.125792] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 5102.435810] Key type lgssc unregistered [ 5102.657894] LNet: 170579:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5102.667305] LNetError: 170579:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5103.723985] LNet: Removed LNI 192.168.201.135@tcp [ 5104.452627] Key type .llcrypt unregistered [ 5104.454608] Key type ._llcrypt unregistered [ 5125.696721] Key type ._llcrypt registered [ 5125.699340] Key type .llcrypt registered [ 5125.787212] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 5126.717798] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5126.786444] alg: No test for adler32 (adler32-zlib) [ 5127.825187] Lustre: Lustre: Build Version: 2.17.57_103_ga9ee299 [ 5128.006420] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 5129.656146] Key type lgssc registered [ 5130.702831] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5137.813345] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5137.826968] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5139.220675] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5143.620074] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5149.672721] LustreError: 171990:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5149.687982] LustreError: 171990:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5152.129967] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5152.505579] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5156.790673] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5163.095928] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5168.030514] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5173.610704] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5173.859528] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5178.559230] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5184.206503] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5185.132277] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5191.062720] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5191.098294] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5191.115869] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5191.131052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.157969] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5191.919973] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5191.943180] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5191.959700] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5191.968744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.976141] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5192.708299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.419422] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5193.425801] Lustre: Skipped 1 previous similar message [ 5193.451783] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5193.459232] Lustre: Skipped 1 previous similar message [ 5193.462360] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5193.466711] Lustre: Skipped 1 previous similar message [ 5193.471995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.485078] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5193.489704] Lustre: Skipped 1 previous similar message [ 5194.371701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.098181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.724768] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5195.727815] Lustre: Skipped 2 previous similar messages [ 5195.738762] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5195.745149] Lustre: Skipped 2 previous similar messages [ 5195.750681] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5195.756240] Lustre: Skipped 2 previous similar messages [ 5195.758899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.766939] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5195.775014] Lustre: Skipped 2 previous similar messages [ 5196.440712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.238664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.029023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.872742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.614832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.327282] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5200.332279] Lustre: Skipped 5 previous similar messages [ 5200.343630] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5200.353747] Lustre: Skipped 5 previous similar messages [ 5200.357481] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5200.365161] Lustre: Skipped 5 previous similar messages [ 5200.373692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.384863] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5200.389491] Lustre: Skipped 5 previous similar messages [ 5201.055590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.801691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.496919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.251312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.955151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.562742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.112339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.835031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.453472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.070905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.763093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.365273] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5208.367898] Lustre: Skipped 11 previous similar messages [ 5208.377244] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5208.382420] Lustre: Skipped 11 previous similar messages [ 5208.385631] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5208.388992] Lustre: Skipped 11 previous similar messages [ 5208.393497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.400837] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5208.404212] Lustre: Skipped 11 previous similar messages [ 5209.031995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.792547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.461739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.140585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.661869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.468258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.070164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.865434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.529553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.320204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.168415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.967694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.568899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.189971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.724462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.358568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.000788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.706352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.468694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.148677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.728744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.368254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.890239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.390749] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5224.394512] Lustre: Skipped 23 previous similar messages [ 5224.402619] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5224.411665] Lustre: Skipped 23 previous similar messages [ 5224.414275] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5224.417045] Lustre: Skipped 23 previous similar messages [ 5224.419213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.431160] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5224.433930] Lustre: Skipped 23 previous similar messages [ 5225.033249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.620312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.256947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.940436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.521530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.297023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.037233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.908511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.684348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.458065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.265981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.119259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.782695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.516558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.118503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.760474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.414929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.138855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.745341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.366428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.024945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.735227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.365819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.961187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.569258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.224981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.834851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.420709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.114531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.758181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.297694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.928599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.576953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.260594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.885417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.530866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.084406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.720471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.406167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.053813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.656147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.363748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.023063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.720705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.460853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.328424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.062755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.882797] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5256.888721] Lustre: Skipped 47 previous similar messages [ 5256.905042] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5256.911632] Lustre: Skipped 47 previous similar messages [ 5256.914806] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5256.918448] Lustre: Skipped 47 previous similar messages [ 5256.921780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.928698] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5256.935164] Lustre: Skipped 47 previous similar messages [ 5257.660816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.395412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.011773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.803485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.444171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.118180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.686621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.348785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.085521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.855524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.491446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.212750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.850668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.471947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.170438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.861640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.556648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.203495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.858844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.557427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.256333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.941657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.584145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.483953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.247267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.014172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.756349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.541136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.337636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.109637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.869471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.611280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.337809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.066206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.887865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.537883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.323791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.071652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.887243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.507668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.339472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.976137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.657143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.374437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.208088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.899681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.613204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.213461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.853866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.486828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.129156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.824488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.527746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.275477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.982392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.793453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.671055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.306931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.998411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.750608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.501574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.157430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.806448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.324902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.987333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.704541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.334486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.097345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.735834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.487653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.224274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.024940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.717718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.380396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.072127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.624981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.291903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.073679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.727465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.318855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.927830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.667201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.357520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.110233] LustreError: 172010:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5316.116474] LustreError: 172010:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5316.132935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.696946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.296773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.905656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.589606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.120329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.751615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.388495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.108486] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5321.113229] Lustre: Skipped 91 previous similar messages [ 5321.125916] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5321.132715] Lustre: Skipped 91 previous similar messages [ 5321.134731] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5321.137019] Lustre: Skipped 91 previous similar messages [ 5321.139317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.146347] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5321.152961] Lustre: Skipped 91 previous similar messages [ 5321.787900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.547064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.124831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.951292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.525403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.183741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.913753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.709047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.435478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.111744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.824039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.495745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.239629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.039901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.655666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.387294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.986914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.691617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.386847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.194644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.945552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.680842] LustreError: 172010:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5336.689548] LustreError: 172010:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5336.697606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.242585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.886651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.541596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.367567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.916324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.558497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.217195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.834265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.376739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.056806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.701921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.365489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.014969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.708456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.361871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.118117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.770389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.401907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.013343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.686031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.317320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.856698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.399149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.999348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.638700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.361895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.060576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.692664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.337715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.215594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.043687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.663499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.465402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.285339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.025118] LustreError: 172010:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5360.039593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.040843] LustreError: 172010:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5360.841572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.705845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.457811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.207723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.844910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.675543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.351768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.230256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.031296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.866257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.484491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.273800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.038951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.876594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.540161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.367559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.017714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.865612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.525817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.311394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.099214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.861333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.615969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.362871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.061337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.845405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.434588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.053386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.712354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.448961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.149060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.907154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.685507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.650581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.345170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.197828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.947612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.703700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.664762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.552917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.294937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.304143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.091551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.903776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.608823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.510168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.247564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.096848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.856115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.686673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.373457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.143180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.908531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.723471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.543855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.343598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.062571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.868650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.702664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.662958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.332179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.254638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.039456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.919827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.793355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.681342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.445618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.288986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.097522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.063123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.988973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.969333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.019972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.959679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.962228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.757269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.750769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.700835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.621361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.393164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.279166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.212146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.165622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.135957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.991408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.768090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.647730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.577744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.438722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.275949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.111980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.939342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.836865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.647305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.507465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.283306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.078793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.815498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.696398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.446425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.208759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.965490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.811271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.555530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.328100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.028690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.731270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.501124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.387467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.102718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.972722] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5449.978578] Lustre: Skipped 167 previous similar messages [ 5449.994787] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5450.000742] Lustre: Skipped 167 previous similar messages [ 5450.004515] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5450.009913] Lustre: Skipped 167 previous similar messages [ 5450.012977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.018620] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5450.026670] Lustre: Skipped 167 previous similar messages [ 5450.857522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.649307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.306463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.022834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.713985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.518225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.272029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.127159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.942065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.862198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.673959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.613898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.327737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.082925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.799396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.561532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.314179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.089849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.955270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.991099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.859680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.642210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.284448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.088411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.857827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.687619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.553645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.440459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.226920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.080490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.779346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.620360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.593551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.715715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.412427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.357910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.098783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.987573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.707466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.549561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.306754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.126850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.896025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.771484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.555768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.517560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.215662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.953627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.565930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.544341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.316563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.025815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.765338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.618540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.303396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.206071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.976835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.791067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.438792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.405851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.178269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.954390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.650393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.455749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.294202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.973157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.591213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.296612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.082947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.006575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.752842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.629912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.385369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.151996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.873236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.576551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.185265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.994291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.821046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.661775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.381207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.146921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.845848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.653857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.340694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.068716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.682200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.317966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.899673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.571749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.257750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.072927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.815441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.828786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.657334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.428270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.242412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.127718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.954044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.764099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.420832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.215250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.015402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.715908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.452979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.237962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.937696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.697277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.415462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.094819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.821789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.653456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.346992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.245869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.058296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.842635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.577788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.199845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.848987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.651766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.309036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.061146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.727689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.435552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.144681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.743414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.461468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.092261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.717840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.334086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.966127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.622125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.316586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.935756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.605291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.197157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.852988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.454917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.168159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.823443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.527782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.333549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.140563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.871587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.655569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.409115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.170603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.952386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.826336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.489588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.301685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.898120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.643519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.292269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.037773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.705961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.384832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.062224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.858695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.501548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.272180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.048305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.880722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.808669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.702423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.533486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.351996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.069986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.064057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.004993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.773439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.538117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.307404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.074127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.924733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.705321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.536911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.207278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.937446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.889731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.698284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.490596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.301553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.112239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.985569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.701949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.477788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.399261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.241757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.058783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.823330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.533482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.461448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.195251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.005214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.782958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.518290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.213864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.901811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.686681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.328090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.099751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.809239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.494084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.181888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.948304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.719913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.388356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.947162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.723766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.468024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.186445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.846467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.639282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.408346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.135450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.824776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.636303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.413781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.325962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.078218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.805812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.647696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.409611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.118289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.862491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.540269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.354607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.079059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.893760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.658193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.309745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.012624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.884237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.511966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.347830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.075576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.869338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.524926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.377459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.072298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.843630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.624747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.463679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.166467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.035106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.845325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.740950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.355775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.028727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.786410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.600131] LustreError: 172010:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5642.607549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.610333] LustreError: 172010:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5643.369164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.309879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.037032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.975428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.777242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.540586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.246858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.974453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.669193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.408895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.093984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.717583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.545454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.151307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.712974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.388298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.962314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.653466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.308651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.918937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.621619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.300412] LustreError: 172010:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5658.304879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.306145] LustreError: 172010:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5659.122455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.918132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.642155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.417977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.990179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.690415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.306470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.081521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.607959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.189144] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5726.193551] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5726.209084] Lustre: Skipped 283 previous similar messages [ 5726.210190] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5726.224541] LustreError: Skipped 1 previous similar message [ 5726.229479] Lustre: Skipped 1 previous similar message [ 5729.841618] Lustre: server umount lustre-OST0000 complete [ 5736.418394] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5736.424867] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5736.425852] Lustre: Skipped 1 previous similar message [ 5738.860547] Lustre: server umount lustre-MDT0000 complete [ 5741.537292] LustreError: 172718:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5741.553650] LustreError: 172718:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 5741.814336] LustreError: 172725:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788267385 with bad export cookie 12641268504490269710 [ 5741.815267] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5741.820420] LustreError: 172725:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5742.174413] Lustre: server umount lustre-MDT0001 complete [ 5751.481928] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 5753.912070] Key type lgssc unregistered [ 5754.183329] LNet: 208622:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5754.189989] LNetError: 208622:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5755.244996] LNet: Removed LNI 192.168.201.135@tcp [ 5755.885803] Key type .llcrypt unregistered [ 5755.888220] Key type ._llcrypt unregistered [ 5777.687418] Key type ._llcrypt registered [ 5777.689926] Key type .llcrypt registered [ 5777.792433] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 5778.833020] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5778.923037] alg: No test for adler32 (adler32-zlib) [ 5779.950571] Lustre: Lustre: Build Version: 2.17.57_103_ga9ee299 [ 5780.176731] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 5781.880210] Key type lgssc registered [ 5782.787711] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5788.894538] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5788.918089] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5790.273579] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5793.560406] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5800.143243] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5800.370725] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5804.358517] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5810.122258] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5815.034674] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5820.774022] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5821.153686] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5826.575766] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5832.939669] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5836.546114] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:10611 to 0x280000401:10657) [ 5842.584633] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5844.169893] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5844.183693] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5844.197291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.214978] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5844.220594] LustreError: 210052:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 5845.987357] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5846.001853] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5846.024522] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5847.015250] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5847.016818] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5847.035508] Lustre: Skipped 1 previous similar message [ 5850.003209] Lustre: lustre-OST0000: Not available for connect from 192.168.201.35@tcp (stopping) [ 5852.132669] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5852.138721] Lustre: Skipped 1 previous similar message [ 5857.249534] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5857.254160] Lustre: Skipped 2 previous similar messages [ 5860.321593] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5860.478459] Lustre: server umount lustre-OST0000 complete [ 5864.248150] LustreError: 210052:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 5894.271138] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5894.519749] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5900.233094] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5906.513859] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5912.949975] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.983768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.985351] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 5917.844658] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 9 sec [ 5922.992828] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8b6351cbf000.ost_server_uuid 50 [ 5924.448959] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8b6351cbf000.ost_server_uuid in FULL state after 0 sec [ 5928.416941] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5928.423294] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5928.436384] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5928.442350] Lustre: Skipped 1 previous similar message [ 5928.455054] Lustre: Skipped 2 previous similar messages [ 5933.719910] Lustre: server umount lustre-OST0000 complete [ 5938.659097] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5938.659686] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5938.676185] Lustre: Skipped 1 previous similar message [ 5942.831361] Lustre: server umount lustre-MDT0000 complete [ 5943.779672] LustreError: 210759:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5943.799124] LustreError: 210759:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 5946.227356] LustreError: 210013:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788267590 with bad export cookie 243392284052936451 [ 5946.238664] LustreError: 210013:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5946.240844] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5946.565227] Lustre: server umount lustre-MDT0001 complete [ 5955.921605] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 5958.198285] Key type lgssc unregistered [ 5958.480708] LNet: 215259:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5958.485643] LNetError: 215259:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5958.502412] LNet: Removed LNI 192.168.201.135@tcp [ 5959.194876] Key type .llcrypt unregistered [ 5959.197596] Key type ._llcrypt unregistered [ 5986.715164] Key type ._llcrypt registered [ 5986.719678] Key type .llcrypt registered [ 5986.821684] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 5987.776086] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5987.916265] alg: No test for adler32 (adler32-zlib) [ 5988.947674] Lustre: Lustre: Build Version: 2.17.57_103_ga9ee299 [ 5989.224243] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 5990.936314] Key type lgssc registered [ 5991.805360] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5999.062490] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5999.073047] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6000.450075] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6003.952883] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6010.456593] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6010.677472] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6014.533777] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6016.879569] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 6021.981337] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6022.264558] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6026.362291] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:10659 to 0x280000401:10689) [ 6026.644339] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6032.412508] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6032.516837] Lustre: lustre-OST0001: new disk, initializing [ 6032.521200] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 6032.527444] Lustre: 219302:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6032.591940] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6034.462130] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 6034.475483] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 6034.556543] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 6036.541421] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6044.395691] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6049.676338] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 6061.051796] Lustre: server umount lustre-MDT0000 complete [ 6061.538854] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6061.550207] LustreError: 216981:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6061.551166] Lustre: Skipped 2 previous similar messages [ 6061.563584] LustreError: 216981:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 6063.298806] LustreError: 216964:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788267707 with bad export cookie 1200008194325562388 [ 6063.300409] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6063.308185] LustreError: 216964:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6063.509883] Lustre: server umount lustre-MDT0001 complete [ 6077.000598] Lustre: server umount lustre-OST0000 complete [ 6089.999683] Lustre: server umount lustre-OST0001 complete [ 6105.850702] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 6112.836390] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6113.102167] LustreError: 222398:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6113.196916] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6116.576284] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6118.370376] LustreError: 222399:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6122.183402] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6122.391231] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6125.282746] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6127.139221] Lustre: 223540:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6131.602123] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6132.899951] LustreError: 223893:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6135.773983] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6138.342127] LustreError: 223896:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6139.385114] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:10691 to 0x280000401:10721) [ 6141.942083] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6142.073842] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6142.076053] Lustre: Skipped 1 previous similar message [ 6145.868748] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6147.600336] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 6151.273907] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6154.267861] Lustre: 225406:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6161.457825] Lustre: server umount mylustre-MDT0000 complete [ 6162.920509] Lustre: mylustre-MDT0000-lwp-OST0000: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6162.933567] Lustre: Skipped 3 previous similar messages [ 6162.935017] LustreError: 222395:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6162.954075] LustreError: 222395:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 6164.086152] LustreError: 222374:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788267807 with bad export cookie 1200008194325565104 [ 6164.089605] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6164.092301] LustreError: 222374:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6166.376327] Lustre: server umount mylustre-MDT0001 complete [ 6178.305945] Lustre: server umount mylustre-OST0000 complete [ 6190.340713] Lustre: server umount mylustre-OST0001 complete [ 6204.353842] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 6211.647448] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6211.989334] LustreError: 227711:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6211.999341] LustreError: 227711:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6212.038673] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6214.945556] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6220.289285] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6223.441323] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6225.397384] Lustre: 228851:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6230.026557] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6235.108587] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6236.427589] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:10691 to 0x280000401:10753) [ 6241.584623] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6245.935373] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6246.907400] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 6250.963714] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6253.484797] Lustre: 230720:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6261.729488] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6261.736523] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6261.749150] Lustre: Skipped 1 previous similar message [ 6261.752406] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 6265.652535] Lustre: server umount tfs-MDT0000 complete [ 6267.371780] LustreError: 227712:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6267.378246] LustreError: 227712:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 8 previous similar messages [ 6267.915482] LustreError: 227687:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788267911 with bad export cookie 1200008194325567001 [ 6267.918771] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6267.923677] LustreError: 227687:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6268.094340] Lustre: server umount tfs-MDT0001 complete [ 6280.798657] Lustre: server umount tfs-OST0000 complete [ 6294.268361] Lustre: server umount tfs-OST0001 complete [ 6308.874995] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 6316.158337] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6316.644774] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6316.650075] Lustre: Skipped 3 previous similar messages [ 6319.958197] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6326.851817] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6330.364481] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6332.342753] Lustre: 234110:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6337.388701] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6341.746415] LustreError: 234464:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6341.759901] LustreError: 234464:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 6341.761121] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:10691 to 0x280000401:10785) [ 6341.913054] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6348.324994] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6351.686270] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 6353.277716] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6358.472986] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6360.813558] Lustre: 235978:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6365.222505] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 09:06:47 (1788268007) [ 6382.571364] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6382.578476] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6382.584069] Lustre: Skipped 5 previous similar messages [ 6382.596614] Lustre: Skipped 7 previous similar messages [ 6385.483966] Lustre: server umount lustre-MDT0000 complete [ 6387.651870] LustreError: 232945:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788268031 with bad export cookie 1200008194325568667 [ 6387.665770] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6387.683437] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6387.684847] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6387.690915] Lustre: Skipped 2 previous similar messages [ 6392.802121] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6392.809415] Lustre: Skipped 2 previous similar messages [ 6393.938793] Lustre: server umount lustre-MDT0001 complete [ 6402.194754] Lustre: server umount lustre-OST0000 complete [ 6410.619333] Lustre: server umount lustre-OST0001 complete [ 6413.560326] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 6417.537059] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 6437.702889] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6437.903446] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6437.926742] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6437.999512] Lustre: lustre-MDT0000: new disk, initializing [ 6438.066633] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6440.720551] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6448.225984] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6448.318666] Lustre: 239158:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6448.344804] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6448.347047] Lustre: Skipped 1 previous similar message [ 6448.393324] Lustre: lustre-MDT0001: new disk, initializing [ 6448.453309] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 188-180 [ 6448.457483] Lustre: Skipped 4 previous similar messages [ 6448.477069] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6448.491671] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6451.564108] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6458.419182] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6458.626406] Lustre: lustre-OST0000: new disk, initializing [ 6458.628466] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6458.634539] Lustre: 240740:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6460.412597] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6460.424668] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6460.522200] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6462.932255] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6469.992346] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6470.118282] Lustre: lustre-OST0001: new disk, initializing [ 6470.121597] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 6470.126000] Lustre: 241611:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6471.588680] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 6471.597345] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 6471.703646] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 6474.544569] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6482.570319] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 09:08:45 (1788268125) [ 6489.897237] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 09:08:52 (1788268132) [ 6506.465975] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6506.471299] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6506.474472] LustreError: Skipped 1 previous similar message [ 6506.476523] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6506.488225] Lustre: Skipped 1 previous similar message [ 6506.495576] Lustre: Skipped 1 previous similar message [ 6508.566262] Lustre: server umount lustre-OST0000 complete [ 6512.610866] LustreError: 240882:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6512.632227] LustreError: 240882:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 6517.729938] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6517.732589] Lustre: Skipped 6 previous similar messages [ 6525.408457] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6525.566695] Lustre: server umount lustre-MDT0000 complete [ 6528.029091] LustreError: 239149:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788268171 with bad export cookie 1200008194325570221 [ 6528.031444] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6528.036496] LustreError: 239149:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 6533.093772] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6533.100294] Lustre: Skipped 3 previous similar messages [ 6534.408898] Lustre: server umount lustre-MDT0001 complete [ 6540.153282] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 6614.262438] Lustre: server umount lustre-OST0001 complete [ 6617.981650] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 6627.135912] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 6644.899441] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6645.067561] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6645.081545] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6645.132512] Lustre: lustre-MDT0000: new disk, initializing [ 6645.181790] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6647.465835] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6653.946450] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6653.997214] Lustre: 246437:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6654.150255] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6656.445281] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6661.632710] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6664.855373] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6668.098209] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6668.235447] Lustre: lustre-OST0000: new disk, initializing [ 6668.237578] Lustre: Skipped 1 previous similar message [ 6668.241501] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6668.244488] Lustre: Skipped 2 previous similar messages [ 6668.247078] Lustre: 248406:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6669.431340] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6669.439084] Lustre: Skipped 1 previous similar message [ 6669.444125] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6669.523519] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6671.541304] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6676.948776] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6685.157265] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6685.162279] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6685.164879] Lustre: Skipped 1 previous similar message [ 6685.165211] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6685.167138] Lustre: Skipped 4 previous similar messages [ 6686.272336] Lustre: server umount lustre-OST0000 complete [ 6690.275971] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6694.734987] Lustre: server umount lustre-MDT0000 complete [ 6696.907959] LustreError: 246475:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788268340 with bad export cookie 1200008194325571691 [ 6696.917096] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6696.923253] LustreError: 246475:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6697.124534] Lustre: server umount lustre-MDT0001 complete [ 6703.301263] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 6704.890732] Key type lgssc unregistered [ 6705.049792] LNet: 250161:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6705.054739] LNetError: 250161:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6705.065472] LNet: Removed LNI 192.168.201.135@tcp [ 6705.535162] Key type .llcrypt unregistered [ 6705.536857] Key type ._llcrypt unregistered [ 6718.872409] Key type ._llcrypt registered [ 6718.874290] Key type .llcrypt registered [ 6718.925540] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 6719.421288] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6719.466092] alg: No test for adler32 (adler32-zlib) [ 6720.375353] Lustre: Lustre: Build Version: 2.17.57_103_ga9ee299 [ 6720.560605] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 6722.208188] Key type lgssc registered [ 6722.897374] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6727.331817] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6727.350199] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6728.601976] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6731.000657] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6735.748263] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6735.960984] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6737.911219] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6741.248935] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6743.804660] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6747.192212] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6747.388695] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6750.237719] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6753.514531] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 6753.882079] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6754.399880] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 6758.882101] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6758.882637] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6758.889249] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6760.936671] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6760.939509] Lustre: Skipped 1 previous similar message [ 6761.542540] Lustre: server umount lustre-OST0000 complete [ 6766.049595] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6766.050560] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6766.055738] Lustre: Skipped 1 previous similar message [ 6769.449827] Lustre: server umount lustre-MDT0000 complete [ 6771.060351] LustreError: 251581:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788268414 with bad export cookie 8877710789400405649 [ 6771.061980] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6771.065045] LustreError: 251581:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6771.211896] Lustre: server umount lustre-MDT0001 complete [ 6775.314746] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 6776.466024] Key type lgssc unregistered [ 6776.618735] LNet: 255180:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6776.624739] LNetError: 255180:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6776.634622] LNet: Removed LNI 192.168.201.135@tcp [ 6777.100927] Key type .llcrypt unregistered [ 6777.102699] Key type ._llcrypt unregistered [ 6785.438653] Key type ._llcrypt registered [ 6785.440390] Key type .llcrypt registered [ 6785.484268] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 6785.886117] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6785.894100] alg: No test for adler32 (adler32-zlib) [ 6786.787569] Lustre: Lustre: Build Version: 2.17.57_103_ga9ee299 [ 6786.900327] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 6788.496201] Key type lgssc registered [ 6788.954428] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6792.833996] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6792.841924] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6808.288385] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 6813.408356] LustreError: 256348:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.135@tcp: failed processing log, type 4: rc = -110 [ 6840.032181] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6842.646079] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6845.653751] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6846.477530] Lustre: server umount lustre-OST0000 complete [ 6852.679826] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 6853.799657] Key type lgssc unregistered [ 6853.953331] LNet: 257976:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6853.955956] LNetError: 257976:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6853.966472] LNet: Removed LNI 192.168.201.135@tcp [ 6854.293151] Key type .llcrypt unregistered [ 6854.294576] Key type ._llcrypt unregistered [ 6871.556259] Key type ._llcrypt registered [ 6871.557494] Key type .llcrypt registered [ 6871.622603] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 6872.047766] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6872.067262] alg: No test for adler32 (adler32-zlib) [ 6872.992706] Lustre: Lustre: Build Version: 2.17.57_103_ga9ee299 [ 6873.133725] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 6874.728215] Key type lgssc registered [ 6875.169727] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6880.780749] loop0: detected capacity change from 0 to 204800000 [ 6880.844279] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6880.848269] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6881.009575] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6881.450438] loop0: detected capacity change from 0 to 204800000 [ 6881.570639] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6881.575320] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6881.745515] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6882.165566] loop0: detected capacity change from 0 to 204800000 [ 6882.272219] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6882.277858] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6882.335398] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6882.811410] loop0: detected capacity change from 0 to 204800000 [ 6882.921868] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6883.395733] loop0: detected capacity change from 0 to 204800000 [ 6883.411740] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6885.076321] loop0: detected capacity change from 0 to 204800000 [ 6885.093220] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6886.672178] loop0: detected capacity change from 0 to 204800000 [ 6886.682494] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6888.250356] loop0: detected capacity change from 0 to 204800000 [ 6888.265541] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6889.820818] loop0: detected capacity change from 0 to 204800000 [ 6889.866022] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6889.882000] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6889.923598] Lustre: 261361:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6889.929983] Lustre: 261361:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 6889.936748] Lustre: 261361:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 6889.940796] Lustre: 261361:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 6889.944806] Lustre: 261361:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 6889.949298] Lustre: 261361:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 6889.953763] Lustre: 261361:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 6889.957858] Lustre: 261361:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 6889.963363] Lustre: 261361:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 6889.968752] Lustre: 261361:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 6889.972843] Lustre: 261361:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 6889.979545] Lustre: 261361:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 6889.985534] Lustre: 261361:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 6889.990436] Lustre: 261361:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 6892.936425] Lustre: server umount MGS complete [ 6893.314896] loop0: detected capacity change from 0 to 204800000 [ 6893.353647] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6893.535705] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6893.553393] Lustre: lustre-MDT0000: Aborting client recovery [ 6893.948535] loop1: detected capacity change from 0 to 204800000 [ 6893.997266] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6894.064130] Lustre: 261792:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6894.068280] Lustre: 261792:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 6894.073274] Lustre: 261792:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 6894.078628] Lustre: 261792:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 6894.083827] Lustre: 261792:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 6894.091275] Lustre: 261792:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 6894.096490] Lustre: 261792:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 6894.108828] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6894.205961] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6894.224221] Lustre: lustre-MDT0001: Aborting client recovery [ 6894.618173] loop2: detected capacity change from 0 to 204800000 [ 6894.662674] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6894.699666] Lustre: 261874:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6894.704987] Lustre: 261874:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 6894.710507] Lustre: 261874:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 6894.716182] Lustre: 261874:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 6894.723100] Lustre: 261874:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 6894.728088] Lustre: 261874:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 6894.735819] Lustre: 261874:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 6894.793643] Lustre: Found index 0 for lustre-OST0000, updating log [ 6894.808199] Lustre: 261892:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6894.850572] Lustre: lustre-OST0000: Aborting recovery [ 6895.284567] loop3: detected capacity change from 0 to 204800000 [ 6895.360822] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6895.380799] Lustre: 261957:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6895.386802] Lustre: 261957:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 6895.392359] Lustre: 261957:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 6895.397616] Lustre: 261957:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 6895.404231] Lustre: 261957:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 6895.410076] Lustre: 261957:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 6895.415402] Lustre: 261957:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 6895.441486] Lustre: 261963:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6895.470163] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6895.472677] Lustre: Skipped 1 previous similar message [ 6895.481342] Lustre: lustre-OST0001: Aborting recovery [ 6900.713366] LustreError: 261736:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -108 [ 6903.781953] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 6903.784357] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 6903.808056] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 6903.808844] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 6904.804756] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 6904.805374] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 6907.245223] LustreError: 261703:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 345us: rc = -2 [ 6920.162975] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6920.166030] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6920.167069] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6920.174311] Lustre: Skipped 2 previous similar messages [ 6920.177265] Lustre: Skipped 1 previous similar message [ 6925.036455] Lustre: server umount lustre-MDT0000 complete [ 6925.281288] LustreError: 261701:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6925.289128] LustreError: 261701:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 6925.461978] LustreError: 261962:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788268569 with bad export cookie 5432920787196844881 [ 6925.465693] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6925.469959] LustreError: 261962:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6930.400923] LustreError: 261702:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6930.401492] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6930.407893] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6930.407902] Lustre: Skipped 2 previous similar messages [ 6930.408234] LustreError: 261702:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6930.424306] Lustre: Skipped 2 previous similar messages [ 6931.845922] Lustre: server umount lustre-MDT0001 complete [ 6938.578797] Lustre: server umount lustre-OST0000 complete [ 6945.122034] Lustre: server umount lustre-OST0001 complete [ 6948.453730] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 09:16:31 (1788268591) [ 6962.072681] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 6966.803042] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 6982.383866] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6982.485793] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6982.505625] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6982.550193] Lustre: lustre-MDT0000: new disk, initializing [ 6982.588111] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6982.595523] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6984.215261] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6989.023795] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6989.068211] Lustre: 266025:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6989.094565] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6989.096494] Lustre: Skipped 1 previous similar message [ 6989.131779] Lustre: lustre-MDT0001: new disk, initializing [ 6989.165664] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6989.178429] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6989.182318] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6990.817076] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 6995.423840] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6997.774274] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7000.482038] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7000.606236] Lustre: lustre-OST0000: new disk, initializing [ 7000.608866] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7000.612125] Lustre: 267990:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7000.647155] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7003.157822] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7006.220927] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7006.228542] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7006.285759] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7007.642667] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7010.470860] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 7017.354741] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 7022.287891] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 7067.619608] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7067.620531] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7067.624782] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7067.626708] LustreError: Skipped 1 previous similar message [ 7067.629392] Lustre: Skipped 2 previous similar messages [ 7072.743679] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7072.751450] Lustre: Skipped 1 previous similar message [ 7077.344127] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7077.408209] Lustre: server umount lustre-OST0000 complete [ 7081.441059] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7081.444540] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7081.451385] Lustre: Skipped 1 previous similar message [ 7081.454377] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7085.358534] Lustre: server umount lustre-MDT0000 complete [ 7087.164469] LustreError: 266017:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788268731 with bad export cookie 5432920787196847597 [ 7087.171620] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7087.173780] LustreError: 266017:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7087.359142] Lustre: server umount lustre-MDT0001 complete [ 7091.512667] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7093.505441] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7095.896338] Lustre: server umount MGS complete [ 7101.257121] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7101.484454] LustreError: 271901:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7101.495233] LustreError: 271914:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 7101.533752] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7103.743917] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7106.530125] LustreError: 271902:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7108.022638] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7110.204785] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7113.906505] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7116.382398] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7119.204195] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7122.021238] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7125.528770] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7157.220148] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7157.223465] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7157.229318] Lustre: Skipped 1 previous similar message [ 7157.234844] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7157.237731] Lustre: Skipped 2 previous similar messages [ 7158.072349] Lustre: server umount lustre-OST0000 complete [ 7164.388182] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7166.208079] Lustre: server umount lustre-MDT0000 complete [ 7167.864584] LustreError: 271883:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788268811 with bad export cookie 5432920787196849410 [ 7167.867657] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7167.870017] LustreError: 271883:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7177.271486] Lustre: server umount lustre-MDT0001 complete [ 7182.718967] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 7184.331900] Key type lgssc unregistered [ 7184.542783] LNet: 275842:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7184.549352] LNetError: 275842:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7184.568917] LNet: Removed LNI 192.168.201.135@tcp [ 7185.122060] Key type .llcrypt unregistered [ 7185.123842] Key type ._llcrypt unregistered [ 7204.130600] Key type ._llcrypt registered [ 7204.132224] Key type .llcrypt registered [ 7204.185655] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 7212.692390] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 7213.241892] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7213.257407] alg: No test for adler32 (adler32-zlib) [ 7214.192229] Lustre: Lustre: Build Version: 2.17.57_103_ga9ee299 [ 7214.330288] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 7215.952185] Key type lgssc registered [ 7216.494293] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7232.938946] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7232.947805] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7234.040594] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7234.054587] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7234.088358] Lustre: lustre-MDT0000: new disk, initializing [ 7234.127349] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7234.135382] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7236.026883] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7241.273681] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7241.322488] Lustre: 279768:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7241.350220] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7241.352433] Lustre: Skipped 1 previous similar message [ 7241.384359] Lustre: lustre-MDT0001: new disk, initializing [ 7241.421136] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7241.448326] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7241.453280] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7242.951553] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7247.404079] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7249.621751] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7252.343655] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7252.461079] Lustre: lustre-OST0000: new disk, initializing [ 7252.463625] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7252.466675] Lustre: 281736:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7252.497204] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7254.801576] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7256.575368] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7256.579715] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7256.627383] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7259.468712] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7263.090623] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 7264.437536] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 7272.018796] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 7307.748400] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7307.749650] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7307.750408] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7307.753681] Lustre: Skipped 1 previous similar message [ 7311.153635] Lustre: server umount lustre-OST0000 complete [ 7312.865319] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7312.866658] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7312.870268] Lustre: Skipped 1 previous similar message [ 7312.872274] Lustre: Skipped 1 previous similar message [ 7317.985370] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7317.990385] Lustre: Skipped 1 previous similar message [ 7318.819390] Lustre: server umount lustre-MDT0000 complete [ 7320.362991] LustreError: 279763:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788268964 with bad export cookie 6245937668857820137 [ 7320.363663] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7320.367028] LustreError: 279763:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7320.514366] Lustre: server umount lustre-MDT0001 complete [ 7323.711728] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7325.273945] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7327.169819] Lustre: server umount MGS complete [ 7331.396834] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7331.566414] LustreError: 285552:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 7331.590877] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7333.086790] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7336.487397] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7336.636960] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7338.272654] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7341.200100] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7343.445117] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7346.284680] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7346.423705] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7348.907660] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7351.776834] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7377.379582] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7377.381964] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7377.385455] LustreError: Skipped 1 previous similar message [ 7377.386611] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7377.389387] Lustre: Skipped 1 previous similar message [ 7382.497396] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7382.500124] Lustre: Skipped 2 previous similar messages [ 7383.001811] Lustre: server umount lustre-OST0000 complete [ 7387.618795] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7390.774632] Lustre: server umount lustre-MDT0000 complete [ 7392.464704] LustreError: 288265:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788269036 with bad export cookie 6245937668857821866 [ 7392.465427] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7392.468755] LustreError: 288265:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7392.624692] Lustre: server umount lustre-MDT0001 complete [ 7397.135813] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 7398.472534] Key type lgssc unregistered [ 7398.648882] LNet: 289478:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7398.652526] LNetError: 289478:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7398.662609] LNet: Removed LNI 192.168.201.135@tcp [ 7399.129803] Key type .llcrypt unregistered [ 7399.131580] Key type ._llcrypt unregistered [ 7407.425055] Key type ._llcrypt registered [ 7407.426407] Key type .llcrypt registered [ 7407.471840] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 7413.811042] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 7414.152131] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7414.187320] alg: No test for adler32 (adler32-zlib) [ 7415.047243] Lustre: Lustre: Build Version: 2.17.57_103_ga9ee299 [ 7415.143765] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 7416.720137] Key type lgssc registered [ 7417.081943] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7425.774128] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 7431.423554] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7431.429944] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7432.514380] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7432.528647] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7432.564898] Lustre: lustre-MDT0000: new disk, initializing [ 7432.587928] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7432.595922] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7434.077343] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7438.551267] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7438.665120] Lustre: lustre-OST0000: new disk, initializing [ 7438.667282] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7438.669679] Lustre: Skipped 1 previous similar message [ 7438.672375] Lustre: 293497:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7438.700746] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7440.817795] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7446.022263] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 7446.025484] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 7446.038148] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 7848.846341] Lustre: server umount lustre-MDT0000 complete [ 7850.969758] LustreError: 292686:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788269494 with bad export cookie 15215307931368854171 [ 7850.974915] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7860.344503] Lustre: server umount lustre-OST0000 complete [ 7878.536694] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 7883.874812] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 7903.545661] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7903.712317] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7903.731851] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7903.811152] Lustre: lustre-MDT0000: new disk, initializing [ 7903.904223] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7903.927518] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7906.898721] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7913.852433] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7913.946704] Lustre: 297764:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7913.986786] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7913.989835] Lustre: Skipped 1 previous similar message [ 7914.045831] Lustre: lustre-MDT0001: new disk, initializing [ 7914.110222] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7914.139897] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7914.145972] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7916.775785] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7923.175305] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7926.689594] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7930.861646] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7931.001926] Lustre: lustre-OST0000: new disk, initializing [ 7931.004185] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7931.011652] Lustre: 299729:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7931.069543] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7934.570172] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7939.078139] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7939.083047] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7939.138226] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7940.031274] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7942.318506] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 7942.437126] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 7944.661662] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 7944.793256] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 7949.284296] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7949.287825] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7949.294196] Lustre: Skipped 1 previous similar message [ 7949.294211] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7949.303445] Lustre: Skipped 1 previous similar message [ 7952.167585] Lustre: server umount lustre-OST0000 complete [ 7955.425262] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7955.428203] LustreError: Skipped 1 previous similar message [ 7955.431837] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7955.438841] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7959.523066] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7959.525153] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7960.705957] Lustre: server umount lustre-MDT0000 complete [ 7962.896972] LustreError: 297757:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788269606 with bad export cookie 15215307931371794962 [ 7962.899481] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7962.901135] LustreError: 297757:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7963.115472] Lustre: server umount lustre-MDT0001 complete [ 7965.158328] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 7966.286672] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 09:33:28 (1788269608) [ 7970.835425] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7971.192271] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7973.851320] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7976.419653] LustreError: 302040:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7976.434882] LustreError: 302040:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7978.784735] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7981.723546] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7986.300867] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7989.711730] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7993.610542] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7993.798064] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7993.802198] Lustre: Skipped 1 previous similar message [ 7997.327394] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8001.393471] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8005.326785] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8005.421813] Lustre: lustre-OST0001: new disk, initializing [ 8005.426784] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8005.433186] Lustre: 304873:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8009.051759] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8011.358071] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8011.362290] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8011.387982] LustreError: 305351:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 8012.451646] LustreError: 305357:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 8012.455423] LustreError: 305357:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 8013.476575] LustreError: 305351:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 8013.481867] LustreError: 305351:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 8014.925639] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 8015.033367] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 8015.523082] LustreError: 305357:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 8015.526862] LustreError: 305357:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 8017.906727] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8b63523e2000.ost_server_uuid 50 [ 8018.631371] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8b63523e2000.ost_server_uuid in FULL state after 0 sec [ 8019.622383] LustreError: 305357:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 8019.635147] LustreError: 305357:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 8021.603586] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8b63523e2000.ost_server_uuid 50 [ 8022.507313] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8b63523e2000.ost_server_uuid in FULL state after 0 sec [ 8023.783262] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8037.349519] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8037.358426] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 8037.359091] Lustre: Skipped 1 previous similar message [ 8037.370342] Lustre: Skipped 2 previous similar messages [ 8042.472389] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 8042.478737] Lustre: Skipped 2 previous similar messages [ 8049.120127] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8049.166354] Lustre: server umount lustre-OST0001 complete [ 8052.708463] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8052.711465] LustreError: 303857:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8052.713142] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8052.713148] Lustre: Skipped 4 previous similar messages [ 8052.716436] Lustre: Skipped 1 previous similar message [ 8052.727575] LustreError: 303857:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8057.830542] LustreError: 303858:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8057.847351] LustreError: 303858:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8057.968612] Lustre: server umount lustre-OST0000 complete [ 8060.898092] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8060.903301] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8066.236432] Lustre: server umount lustre-MDT0000 complete [ 8068.065238] LustreError: 302767:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8068.072162] LustreError: 302767:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8068.407892] LustreError: 302019:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788269712 with bad export cookie 15215307931371795844 [ 8068.410586] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8068.414248] LustreError: 302019:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8068.641986] Lustre: server umount lustre-MDT0001 complete [ 8074.824741] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 8076.496494] Key type lgssc unregistered [ 8076.727349] LNet: 307554:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8076.730376] LNetError: 307554:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8076.740689] LNet: Removed LNI 192.168.201.135@tcp [ 8077.336618] Key type .llcrypt unregistered [ 8077.338759] Key type ._llcrypt unregistered [ 8092.556356] Key type ._llcrypt registered [ 8092.557778] Key type .llcrypt registered [ 8092.618603] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 8093.170756] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8093.305461] alg: No test for adler32 (adler32-zlib) [ 8094.259150] Lustre: Lustre: Build Version: 2.17.57_103_ga9ee299 [ 8094.454641] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 8096.080177] Key type lgssc registered [ 8096.662503] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8101.054934] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8101.066492] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8102.293673] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8104.695720] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8108.676925] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8112.612676] LustreError: 308949:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8112.623135] LustreError: 308949:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8112.641966] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8112.853657] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8114.910598] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8120.470234] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8120.722498] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8122.791604] LustreError: 310842:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8124.089515] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8127.913031] LustreError: 310842:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8127.926474] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 8127.929478] LustreError: 310842:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8128.563442] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8131.907423] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8132.052456] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8134.955848] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8137.206421] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 8138.532504] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8142.638786] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 8142.742820] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 8229.348183] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8229.348261] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8229.353820] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 8229.357455] Lustre: Skipped 1 previous similar message [ 8233.542514] Lustre: server umount lustre-OST0001 complete [ 8234.470886] LustreError: 312196:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8234.477022] LustreError: 312196:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8239.587987] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8239.589775] LustreError: 312196:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8239.590649] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8239.590654] Lustre: Skipped 1 previous similar message [ 8239.600113] Lustre: Skipped 1 previous similar message [ 8239.614641] LustreError: 312196:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8244.708350] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8244.712923] Lustre: Skipped 2 previous similar messages [ 8249.828784] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8249.832778] LustreError: 312196:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8249.841738] LustreError: 312196:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8250.336102] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8250.433683] Lustre: server umount lustre-OST0000 complete [ 8254.946077] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8254.948286] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8254.953206] Lustre: Skipped 1 previous similar message [ 8254.956334] Lustre: Skipped 1 previous similar message [ 8258.580369] Lustre: server umount lustre-MDT0000 complete [ 8260.625567] LustreError: 308929:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788269904 with bad export cookie 14605097765123934349 [ 8260.628685] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8260.631921] LustreError: 308929:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8260.806881] Lustre: server umount lustre-MDT0001 complete [ 8267.975794] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 8269.660258] Key type lgssc unregistered [ 8269.819694] LNet: 313890:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8269.825556] LNetError: 313890:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8269.836592] LNet: Removed LNI 192.168.201.135@tcp [ 8270.318220] Key type .llcrypt unregistered [ 8270.319669] Key type ._llcrypt unregistered [ 8298.039532] Key type ._llcrypt registered [ 8298.041445] Key type .llcrypt registered [ 8298.102925] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 8298.759732] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8298.774458] alg: No test for adler32 (adler32-zlib) [ 8299.702417] Lustre: Lustre: Build Version: 2.17.57_103_ga9ee299 [ 8299.824125] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 8301.432333] Key type lgssc registered [ 8301.973653] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8306.344835] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8306.357944] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8307.617214] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8310.399617] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8317.611550] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8317.944644] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8320.802832] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8323.011539] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 8328.705961] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8329.009845] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8333.121855] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8334.119592] LustreError: 317642:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8334.127910] LustreError: 317642:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8334.134505] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 8338.026366] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8338.183933] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8341.218476] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8342.326724] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 8347.861685] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8352.661542] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 8358.881366] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8358.885630] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8358.891585] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8362.980306] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8362.984401] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8362.990539] Lustre: Skipped 2 previous similar messages [ 8362.999879] Lustre: Skipped 3 previous similar messages [ 8364.550977] Lustre: server umount lustre-MDT0000 complete [ 8366.130550] LustreError: 319159:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788270009 with bad export cookie 4374111693143209618 [ 8366.133818] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8366.135838] LustreError: 319159:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8366.303443] Lustre: server umount lustre-MDT0001 complete [ 8377.302139] Lustre: server umount lustre-OST0000 complete [ 8389.624422] Lustre: server umount lustre-OST0001 complete [ 8398.911482] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 8404.499227] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8404.738935] LustreError: 321413:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8404.792770] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8407.091266] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8410.082220] LustreError: 321414:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8411.693167] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8414.517789] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8416.418360] Lustre: 322551:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8420.133358] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8420.370764] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8420.375810] Lustre: Skipped 1 previous similar message [ 8421.415967] LustreError: 322903:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8423.450343] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8426.473226] LustreError: 322905:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8428.057546] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8429.236892] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 8429.251289] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 8431.603709] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8436.638269] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8438.967949] Lustre: 324422:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8442.849469] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8442.853224] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8442.859802] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8444.900294] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8444.907453] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8444.916491] Lustre: Skipped 2 previous similar messages [ 8444.920133] Lustre: Skipped 2 previous similar messages [ 8447.103485] Lustre: server umount lustre-MDT0000 complete [ 8449.234814] LustreError: 321394:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788270093 with bad export cookie 4374111693143211319 [ 8449.238146] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8449.247065] LustreError: 321394:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8449.391574] Lustre: server umount lustre-MDT0001 complete [ 8461.901322] Lustre: server umount lustre-OST0000 complete [ 8474.612657] Lustre: server umount lustre-OST0001 complete [ 8476.736815] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 8477.719324] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 8478.706359] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 09:42:01 (1788270121) [ 8491.094740] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 8496.257640] loop0: detected capacity change from 0 to 1099511627776 [ 8496.604634] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 8496.609960] *** VALIDATE xfs *** [ 8496.621608] XFS (loop0): Mounting V5 Filesystem [ 8496.649298] XFS (loop0): Ending clean mount [ 8497.364511] loop1: detected capacity change from 0 to 18691697672192 [ 8537.621187] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 8542.841471] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 8561.867102] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8562.046158] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8562.062655] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8562.107645] Lustre: lustre-MDT0000: new disk, initializing [ 8562.149306] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8562.151257] Lustre: Skipped 1 previous similar message [ 8562.160751] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8564.327697] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8570.525442] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8570.581668] Lustre: 329539:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8570.614677] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8570.617556] Lustre: Skipped 1 previous similar message [ 8570.661275] Lustre: lustre-MDT0001: new disk, initializing [ 8570.737617] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8570.743058] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8572.623119] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8577.542420] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8580.108936] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8583.804763] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8583.978720] Lustre: lustre-OST0000: new disk, initializing [ 8583.984805] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8583.991182] Lustre: 331508:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8585.167985] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8585.175949] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8585.231135] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8587.169148] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8592.313124] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8594.516915] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 8594.630231] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 8596.907466] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 8597.048707] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 8600.553474] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8600.556263] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8600.558787] LustreError: Skipped 1 previous similar message [ 8600.560251] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8600.564832] Lustre: Skipped 1 previous similar message [ 8600.580076] Lustre: Skipped 1 previous similar message [ 8610.788049] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8610.793831] Lustre: Skipped 3 previous similar messages [ 8612.320126] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8612.415056] Lustre: server umount lustre-OST0000 complete [ 8615.907331] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8615.918721] Lustre: Skipped 1 previous similar message [ 8620.568853] Lustre: server umount lustre-MDT0000 complete [ 8621.028361] LustreError: 329550:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8621.038993] LustreError: 329550:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8622.386801] LustreError: 330336:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788270266 with bad export cookie 4374111693143212866 [ 8622.387300] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8622.392213] LustreError: 330336:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8622.563056] Lustre: server umount lustre-MDT0001 complete [ 8626.435156] XFS (loop0): Unmounting Filesystem [ 8628.621653] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 09:44:31 (1788270271) [ 8631.951722] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8632.218599] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8632.223531] Lustre: Skipped 2 previous similar messages [ 8634.097694] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8637.416788] LustreError: 334062:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8637.423166] LustreError: 334062:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8638.202104] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8640.428339] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8643.981583] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8646.623149] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8650.305952] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8653.286225] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8656.643247] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8674.277540] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8674.281573] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8674.286636] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8674.298161] Lustre: Skipped 1 previous similar message [ 8674.309335] Lustre: Skipped 3 previous similar messages [ 8680.326693] Lustre: server umount lustre-OST0000 complete [ 8692.731588] Lustre: server umount lustre-MDT0000 complete [ 8694.758659] LustreError: 336501:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8694.788507] LustreError: 336501:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8696.787395] LustreError: 334043:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788270340 with bad export cookie 4374111693143213748 [ 8696.805422] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8696.811038] LustreError: 334043:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8697.245966] Lustre: server umount lustre-MDT0001 complete [ 8711.101224] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 8714.194661] Key type lgssc unregistered [ 8714.519628] LNet: 337709:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8714.524837] LNetError: 337709:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8714.537908] LNet: Removed LNI 192.168.201.135@tcp [ 8715.353154] Key type .llcrypt unregistered [ 8715.354470] Key type ._llcrypt unregistered [ 8739.955951] Key type ._llcrypt registered [ 8739.957972] Key type .llcrypt registered [ 8740.040295] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 8740.715175] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8740.897672] alg: No test for adler32 (adler32-zlib) [ 8741.874687] Lustre: Lustre: Build Version: 2.17.57_103_ga9ee299 [ 8742.071524] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 8743.720176] Key type lgssc registered [ 8744.512526] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8750.395142] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8750.418265] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8751.735328] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8754.880991] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8761.197897] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8761.313126] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 8761.410717] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8764.477858] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8770.391834] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8774.265778] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8779.600857] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8779.824241] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8784.222238] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8788.860748] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8804.745309] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.35@tcp (stopping) [ 8806.370367] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8806.387698] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8806.391111] Lustre: Skipped 1 previous similar message [ 8809.858042] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.35@tcp (stopping) [ 8810.653793] Lustre: server umount lustre-MDT0001 complete [ 8811.490843] LustreError: 340363:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8811.509728] LustreError: 340363:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8814.978959] LustreError: 339264:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.35@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8816.612387] LustreError: 339268:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8816.637424] LustreError: 339268:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8818.193318] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8818.258398] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 8818.261251] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8818.279593] Lustre: 339256:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8818.468685] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8821.472771] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8823.792209] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 8827.905729] LustreError: 342917:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 8827.913582] LustreError: 342917:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 8827.919928] LustreError: 339260:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 8827.964422] LustreError: 342925:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.135@tcp: cfg command failed: rc = -17 [ 8827.972564] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 8827.972564] [ 9127.318440] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.35@tcp (stopping) [ 9130.979990] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 9130.984718] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9130.998017] Lustre: Skipped 2 previous similar messages [ 9136.099609] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9136.106141] Lustre: Skipped 4 previous similar messages [ 9141.216121] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9141.448437] Lustre: server umount lustre-MDT0001 complete [ 9142.655924] LustreError: 339264:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.35@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9147.773590] LustreError: 339265:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.35@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9147.790831] LustreError: 339265:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 9148.651344] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9148.717078] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 9148.722368] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9148.748078] Lustre: 339258:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9148.968269] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9151.698018] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 9154.031960] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 9156.590254] LustreError: 352688:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 9156.596841] LustreError: 352688:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 9156.602852] LustreError: 339260:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 9156.608250] LustreError: 339260:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 9156.654258] LustreError: 352691:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [ 9156.659699] LustreError: 352691:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [ 9156.664988] LustreError: 352691:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.135@tcp: cfg command failed: rc = -17 [ 9156.671994] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.201.135@tcp [ 9156.671994] [ 9454.980455] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.35@tcp (stopping) [ 9454.983588] Lustre: Skipped 3 previous similar messages [ 9456.099126] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9456.099387] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 9460.693865] Lustre: server umount lustre-MDT0001 complete [ 9461.218176] LustreError: 355700:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9461.223867] LustreError: 355700:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9464.595717] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9464.635362] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 9464.637644] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9464.651725] Lustre: 339257:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9464.770097] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 9466.274347] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 9469.925289] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 9470.444341] LustreError: 364115:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 9470.447912] LustreError: 364115:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 9470.451774] LustreError: 339260:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 9470.458887] LustreError: 339260:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 9470.503704] LustreError: 364124:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 9470.514796] LustreError: 364124:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.135@tcp: cfg command failed: rc = -114 [ 9470.521727] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 9470.521727] [ 9782.241657] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9782.244570] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9782.249581] Lustre: Skipped 1 previous similar message [ 9782.251960] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9782.254483] Lustre: Skipped 3 previous similar messages [ 9787.057329] Lustre: server umount lustre-MDT0000 complete [ 9789.128409] LustreError: 375423:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788271432 with bad export cookie 10939334007850839385 [ 9789.134590] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9789.137115] LustreError: 375423:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9790.433328] LustreError: 339263:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9790.433738] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9790.447774] Lustre: Skipped 2 previous similar messages [ 9795.390342] Lustre: server umount lustre-MDT0001 complete [ 9797.845768] Lustre: server umount lustre-OST0000 complete [ 9805.506897] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 9814.312302] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 9838.777323] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 10:04:41 (1788271481) [ 9842.272195] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9842.416455] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9842.454916] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9842.558712] Lustre: lustre-MDT0000: new disk, initializing [ 9842.641060] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9842.664088] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9844.956059] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 9850.482995] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9850.528367] Lustre: 378482:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9850.561428] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9850.564570] Lustre: Skipped 1 previous similar message [ 9850.600479] Lustre: lustre-MDT0001: new disk, initializing [ 9850.643352] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9850.661668] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9850.666567] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9852.508588] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 9857.400856] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9860.401816] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9865.242454] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9865.532518] Lustre: lustre-OST0000: new disk, initializing [ 9865.538496] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9865.555324] Lustre: 380448:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9865.674489] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9869.765692] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 9873.515725] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9873.537613] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9873.697313] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9877.290721] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9886.689543] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9886.700182] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9886.717240] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9886.722155] Lustre: Skipped 5 previous similar messages [ 9888.282149] Lustre: server umount lustre-MDT0000 complete [ 9889.254034] LustreError: 380872:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9889.269812] LustreError: 380872:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9892.473355] LustreError: 379286:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788271536 with bad export cookie 10939334007850841338 [ 9892.479774] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9892.484019] LustreError: 379289:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [ 9892.485211] LustreError: 379286:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9892.835545] Lustre: server umount lustre-MDT0001 complete [ 9907.903853] Lustre: server umount lustre-OST0000 complete [ 9960.527353] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 9973.347521] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [10004.352793] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 9714 sec ========== 10:07:26 (1788271646) [10005.917745] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 10:07:27 (1788271647) === [10009.037204] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 10:07:31 (1788271651) === [10046.433377] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [10049.148344] Key type lgssc unregistered [10049.432624] LNet: 386815:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10049.438146] LNetError: 386815:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10049.455566] LNet: Removed LNI 192.168.201.135@tcp [10050.324126] Key type .llcrypt unregistered [10050.335428] Key type ._llcrypt unregistered